Reviewing Coveo Machine Learning Query Suggestion Candidates

The Coveo Machine Learning (Coveo ML) Query Suggestions (QS) feature proposes the most relevant similar queries that were performed before.

To prevent infrequent queries to pollute the suggestions, the feature delivers suggestions only for queries that were performed and followed by at least one click on search results enough times. The required number of times the query must be performed and followed by a click varies based on the number of queries per search hub (origin level 1), search interface (origin level 2), and language in the model. The query count is relative to the model data period (e.g., the number of queries that were performed in the last month).

The following table lists diverse query counts and the respective minimum number of times a specific query must be performed and followed by a click for this query to be selected as a potential suggestion.

Query count Minimum number of times a given query must be performed and followed by a click1
0 0
1 to 999 1
1,000 to 39,999 2
40,000 to 149,999 3
150,000 to 399,999 4
400,000 to 899,999 5
900,000 and more 6

1: The minimums are calculated, among other things, per language, therefore more popular languages will have a higher minimum number of times. You can review the minimum for each language in the model information panel (see Reviewing Coveo Machine Learning Model Information).

Before enabling Coveo ML QS, you may wonder which queries logged by Coveo Usage Analytics (Coveo UA) from a given search interface would be candidates for query suggestions, basically ensuring that there are queries matching the minimum requirements. This verification is particularly useful when your search interface has low traffic, such as in the case of a search interface only accessible to employees in a small company.

You can use the Analytics section of the Administration Console navigation menu to list queries that match the minimal QS requirements. You can then establish whether the feature can have a significant impact on a specific search interface for a specific language.

The following procedure assumes that you’re familiar with global dimension filters and dashboards (see Add Global Dimension Filters and Review and Manage Dashboards).

Review Query Suggestion Candidates

This section contains a procedure that shows how to create a dashboard displaying the queries that meet the requirements to be suggested by a Coveo ML QS model for a specific search interface (i.e., queries that were performed and followed by a click the required number of times).

If you want to create the dashboard using the Usage Analytics Read API, see Review Query Suggestion Candidates Using the Usage Analytics Read API.

You may want to report on the suggested queries that have been clicked by users. You can do so by reporting on the search events for which the Event Cause has the omniboxAnalytics value.

  1. On the Reports page, click Add, and then select Blank dashboard.

  2. In the empty report that appears, in the upper-left corner of the screen, enter a title for the dashboard (e.g., Queries Suggested by Coveo ML).

  3. (Optional) Enter a dashboard description.

  4. Select a date interval of three months (see Set the Period to Review Search Usage Data).

  5. Add the following global filters (see Add Global Dimension Filters):

    • User Query is not blank or n/a

    • Origin 1 (page/hub) is <SEARCH_PAGE_OR_SEARCH_HUB_NAME>

    • Origin 2 (tab/interface) is <TAB_OR_SEARCH_INTERFACE_NAME>

    • Language is <LANGUAGE>

    Where you replace:

  6. Click Add card to section, and then add a Metric card with the Search Event Count metric (see Add Metric Cards in Usage Analytics Dashboards).

  7. Click Add card to section, and then add a Table card with the following dimension and metric (see Add Table Cards in Usage Analytics Dashboards):

    • Dimensions: User Query

    • Metrics: Click Event Count

  8. Based on the Search Event Count value (obtained in the previous step) and the reference table at the beginning of the article, add the following metric filter: Click Event Count Is greater than or equal to <N>

    Where you replace <N> with the minimum number of clicks as found in the reference table.

    For example, if your search page has received 152 235 queries in English, you must add Click Event Count Is greater than or equal to 3 as the metric filter.

    If you already have a QS model, you can access the model information panel to see the exact minimum number of times that the query must be performed and followed by a click for a specific language (see Reviewing Coveo Machine Learning Model Information).

  9. Save the dashboard.

  10. Browse the table card and see how many queries match the QS feature criteria.

    Ensure that your search bar doesn’t show an empty JavaScript Search Omnibox, when no query suggestions are available.

When your are done creating your dashboard, it should look like this:

report example

Review Query Suggestion Candidates Using the Usage Analytics Read API

The following JSON shows a configuration that you can use to create the above report using the Usage Analytics Read API (see Reports API - Version 15 - Create a report):

{
 "displayName": "<DASHBOARD_NAME>",
 "type": "DASHBOARD",
 "configuration": {
  "description": "<DESCRIPTION>",
  "dateRange": {
   "range": "days",
   "length": 91,
   "offset": -91
  },
  "compareRange": {
   "range": "days",
   "length": 91,
   "offset": -182
  },
  "tabs": [
   {
    "title": "<TAB_NAME>",
    "sections": [
     {
      "title": "",
      "position": {
       "sizex": "6",
       "sizey": "4",
       "col": "1",
       "row": "1"
      },
      "cards": [
       {
        "title": "<METRIC_CARD_TITLE>",
        "cardType": "QuickMetric",
        "eventType": "join",
        "href": "",
        "headerHref": "",
        "position": {
         "minSizex": 1,
         "minSizey": 1,
         "sizex": 1,
         "sizey": 1,
         "col": 1,
         "row": 1
        },
        "metric": "PerformSearch",
        "showTrend": false,
        "exceedsLicense": false,
        "valueType": "total",
        "filters": "",
        "bindOnLastSearch": true
       },
       {
        "title": "<TABLE_CARD_TITLE>",
        "cardType": "DetailedStatistics",
        "eventType": "join",
        "href": "",
        "headerHref": "",
        "position": {
         "minSizex": 1,
         "minSizey": 2,
         "sizex": 1,
         "sizey": 3,
         "col": 2,
         "row": 1
        },
        "filters": "",
        "metricFilters": "(DocumentView>=<MIN_VALUE>)",
        "bindOnLastSearch": true,
        "showCount": false,
        "showHeader": true,
        "sortBy": "DocumentView",
        "ascending": false,
        "dimensionsHeaders": {},
        "metricsHeaders": {},
        "metrics": [
         "DocumentView"
        ],
        "dimensions": [
         "queryExpression"
        ],
        "metricsSort": [
         "DocumentView"
        ],
        "dimensionsSort": [
         "queryExpression"
        ]
       }
      ],
      "editMode": true
     }
    ],
    "active": true,
    "editMode": false
   }
  ],
  "filters": "(queryexpression!='' AND queryexpression!=null) AND (originlevel1=='<ORIGIN_1_VALUE>') AND (originlevel2=='<ORIGIN_2_VALUE>') AND (language=='<LANGUAGE>')",
  "version": 6
 },
 "allAnalyticsViewer": true,
 "filters": [
  ""
 ]
}

Where you replace:

  • <DASHBOARD_NAME> with a title for the dashboard (e.g., Queries Suggested by Coveo ML)

  • <DESCRIPTION> with a description for the dashboard

  • <TAB_NAME> with a name for the report tab on which the dashboard will be displayed

  • <METRIC_CARD_TITLE> with a name for the metric card that displays the total Search Event Count

  • <TABLE_CARD_TITLE> with a name for the table card that displays the queries that meet the requirements to be suggested by Coveo ML

  • <MIN_VALUE> with the minimum number of clicks a query must have to appear as a query candidate (as found in the reference table)

  • <ORIGIN_1_VALUE> with the value of the desired Origin 1 (Page/Hub) (formerly Search Hub)

  • <ORIGIN_2_VALUE> with the value of the desired Origin 2 (Tab) (Formerly Search Interface)

  • <LANGUAGE> with the desired ISO 639-1 language code (e.g., en)

Pushing Query Suggestions

When relevant queries are missing, consider adding them by manually (or automatically with a script) generating usage analytics events (the queries must be sent with three different visit IDs):

  1. In the browser of your choice, in private mode, access your Coveo-powered search interface.

  2. Without being authenticated, perform a query that you want your Coveo ML model to suggest.

  3. Click a search result title.

  4. Close your browser.

  5. Repeat this procedure three times for each query.

  6. After the model is retrained, these suggestions will become available.

Required Privileges

The following table indicates the required privileges to view and edit dashboards from the Reports page and associated panels (see Manage Privileges and Privilege Reference).

Access to dashboards or part of their content may be further restricted as a function of the member (see Manage Access to Usage Analytics Reports and Manage Permission Filters).

Action Service - Domain Required access level
View reports

Analytics - Analytics data

Analytics - Reports

View
Add, edit, and delete reports

Analytics - Analytics data

View

Analytics - Reports

Edit
Analytics - Administrate Allowed
Recommended Articles