Logging Search Events

Typically, when the end user interacts with the search interface in a way that triggers a query to the Search API (e.g., submitting a query from the search box, selecting a facet value, etc.), the search interface logs a search event. This is by far the most commonly logged category of usage analytics event.

This article provides guidelines for performing HTTP requests against the Usage Analytics Write API to properly log search events.

The information in this article is only relevant when you cannot (or choose not to) use the Coveo JavaScript Search Framework to achieve a search integration with the Coveo Cloud platform. Otherwise, the framework will handle all HTTP requests on its own, assuming that an Analytics component is present in the search page.

To log a search event, make a POST request to https://platform.cloud.coveo.com/rest/ua/v15/analytics/search (or https://usageanalyticshipaa.cloud.coveo.com/rest/v15/analytics/search in the HIPAA environment, or https://usageanalytics.coveo.com/rest/v15/analytics/search on Coveo Cloud V1). Use an access token with the privilege to push analytics data (i.e., the Allowed access level on the Analytics data domain) in the target Coveo Cloud organization to authenticate the HTTP request (see Privilege Management and Analytics Data Domain).

A log search event call in which only the body properties which are required by the Coveo usage analytics and ML services are included

POST https://platform.cloud.coveo.com/rest/ua/v15/analytics/search?visitor=28s6g49d-f81s-1435-2r5x153dle72 HTTP/1.1
 
Content-Type: application/json
Cookie: visitor=28s6g49d-f81s-1435-2r5x153dle72
Authorization: Bearer **********-****-****-****-************

Payload

{
  "anonymous": false,
  "language": "en",
  "originLevel1": "CommunitySearchPage",
  "originLevel2": "All",
  "actionCause": "searchboxSubmit",
  "queryText": "coveo",
  "responseTime": 145,
  "searchQueryUid": "7bfc652a-9dea-4811-b3f9-6d24345c37ce",
  "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36"
}

In this example, the username is not included, although it is required, since it is extracted from the search token. If another authentication method was used, the username field would need to be included in the request body.

There is also a method for logging search events in batches (POST https://platform.cloud.coveo.com/rest/ua/v15/analytics/searches, see Add many search events via POST). Assuming that the search interface is logging usage analytics events asynchronously, it could queue up events and send them in a batch when they are ready, which would allow the search interface to make fewer HTTP requests (especially when many search requests that log search events are performed in a short amount of time, e.g., when search-as-you-type is used).

Required Request Body Properties

The following properties are required by the Usage Analytics Write API when logging a search event:

actionCause (string)

The identifier of the end-user action that triggered a query and caused the search interface to log a search event (e.g., if the end user submits a query from the search box, the standard actionCause value to use when logging a search event would be searchboxSubmit). Most actions which can be performed from a typical search interface are associated to a specific, standard actionCause value (i.e., the actionCause that the Coveo JavaScript Search Framework would use in a similar case). Some of the most common actionCause values are documented in the search interface implementation guidelines articles (see Implementing a Search Interface). The full list of all standard actionCause values is available directly in the Coveo JavaScript Search Framework source code (see analyticsActionCauseList).

Some actionCause values are expected to be accompanied by a specific set of customData key-value pairs (see analyticsActionCauseList). Passing these expected key-value pairs is important since they are used by ML and usage analytics reports.

The Coveo Machine Learning (Coveo ML) Query Suggestions (QS) models will learn from a given Usage Analytics Write log request when actionCause is set to one of the following: omniboxAnalytics, searchboxSubmit, searchFromLink.

{
  ...
  "actionCause": "facetSelect",
  ...
}

language (string)

The language of the search interface from which the search event originates. Value must be an ISO 639-1 code. Must match the first part of the locale value in the request body of the corresponding query to the Search API.

The Coveo ML Automatic Relevance Tuning (ART) and Query Suggestions (QS) models handle the query string differently depending on the language value. Additionally, QS models only provide suggestions in the appropriate language, and Event Recommendations (ER) models are language-specific.

{
  ...
  "language": "en",
  ...
}

queryText (string)

The original basic query expression (q) in the corresponding search request.

The queryText value must be the same as the q value of the corresponding search request to the Search API.

The queryText value is used by Coveo ML ART, QS, and ER. It is broken down and associated with relevant result items.

{
  ...
  "queryText": "coveo",
  ...
}

responseTime (strictly positive integer)

The amount of time (in milliseconds) between the moment the query that caused the search interface to log a search event was sent to the Search API and the moment the search interface received the results. Useful to monitor the speed of your solution.

{
  ...
  "responseTime": 145,
  ...
}

searchQueryUid (string)

The unique identifier of the query that caused the search interface to log a search event.

The searchQueryUid value must be the same as the searchUid value in the response of the corresponding search request to the Search API.

The Coveo ML ART and QS models use searchQueryUid to relate click events to search events.

{
  ...
  "searchQueryUid": "7bfc652a-9dea-4811-b3f9-6d24345c37ce",
  ...
}

Coveo ML-Required Request Body Properties

Although optional from the Usage Analytics Write API point of view, the following properties are required by the Coveo ML service when logging a search event (otherwise, Coveo ML models will not function properly):

anonymous (boolean)

Whether the query that caused the search interface to log a search event was triggered by an anonymous user. Default value is false.

If the end user is authenticated, but wants to be anonymous, set anonymous to true. The Usage Analytics Write API will not extract the name and userDisplayName, if present, from the search token. As long as you do not attempt to include the end-user identity anywhere else in your request, the end user will remain anonymous.

The anonymous value must be the same as the isGuestUser value of the corresponding search request to the Search API. If an authenticated user wants to be anonymous, isGuestUser should therefore be set to true when querying the Search API.

When anonymous is set to true, the Coveo ML QS models will personalize using visitorId instead of username.

{
  ...
  "anonymous": false,
  ...
}

userAgent (string)

Information about the browser and operating system of the end-user who triggered the query that caused the search interface to log a search event.

If the client is performing HTTP requests directly against the Usage Analytics Write API to log usage analytics events (as opposed performing those requests through a proxy server), the end-user browser will typically include the correct information in the User-Agent request header. When this is the case, specifying a value for the userAgent request body property is unnecessary.

The Coveo ML ART, QS, and ER models use the userAgent value to filter out search events logged by bots or anonymous users.

{
  ...
  "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36",
  ...
}

username (string)

The name of the user security identity who triggered the query that caused the search interface to log a search event. Can only be specified when authenticated with an API key or an OAuth2 token with the privilege to impersonate users (i.e., the Allowed access level on the Impersonate domain) (see Privilege Management and Impersonate Domain).

Of course, an API key or an OAuth2 token with the privilege to impersonate users should not be exposed publicly (see Choosing and Implementing a Search Authentication Method).

If unspecified and the request is authenticated with a search token, the service attempts to extract the username value from the access token that authenticated the log search event request. If this fails, the username value is left blank.

If unspecified and the request is authenticated with an API key, username is set to platform@coveo.com.

The Coveo ML service requires the username value to load even though it is not always actually used. The Coveo ML service uses the username in query suggestions to filter out irrelevant users, and in event recommendations to build user profiles as well as to provide user based recommendations.

{
  ...
  "username": "alice.smith@example.com",
  ...
}

Coveo ML-Optional Request Body Properties

The following properties are optional from both the Usage Analytics Write API and Coveo ML service points of view when logging search events. However, specifying values for those properties may improve the effectiveness of Coveo ML.

originLevel1 (string)

The name/identifier of the search interface from which the search event originates. If unspecified and the request is authenticated with a search token, the service attempts to extract the searchHub value from the access token that authenticated the log search event request.

When logging a search event, the originLevel1 value must be the same as the searchHub value in the request body of the corresponding query to the Search API.

Coveo ML models use the originLevel1 value as a default filter field. Filters do not impact the learning part of Coveo ML models, but it is important to log them correctly for recommendations to work. When the Coveo ML models make suggestions, items which do not meet the right filters are ignored (see also Getting Query Suggestions - Troubleshooting).

{
  ...
  "originLevel1": "ExternalSearch",
  ...
}

originLevel2 (string)

The name/identifier of the tab from which the search event originates.

When logging a search event, the originLevel2 value must be the same as the tab value in the request body of the corresponding query to the Search API. If there is no tab in your search interface, you can reuse the searchHub value to fulfill the requirement. In other words, you can use the same value for originLevel1 and originLevel2.

Coveo ML models use the originLevel2 value as a default filter field. Filters do not impact the learning part of Coveo ML models, but it is important to log them correctly for recommendations to work. When the Coveo ML models make suggestions, items which do not meet the right filters are ignored.

{
  ...
  "originLevel2": "All",
  ...
}

outcome (integer from -5 to 5)

An indication of how good the outcome of this event is. A value of -5 corresponds to the worst possible outcome, a value of 0 corresponds to a neutral outcome, and a value of 5 corresponds to the best possible outcome.

The outcome property will eventually be leveraged by the Coveo ML service.

{
  ...
  "outcome": 0,
  ...
}

Optional Request Body Properties

The following properties are entirely optional when logging search events. They are not leveraged in any way by the Coveo ML service. Nevertheless, specifying values for those properties can be useful for reporting purposes.

advancedQuery (string)

The original advanced query expression (aq) in the corresponding search request.

If specified, the advancedQuery value must be the same as the advanced query expression (aq) value of the corresponding search request to the Search API.

{
  ...
  "advancedQuery": "(@objecttype==('Developer Guide', 'Developers Topic'))",
  ...
}

numberOfResults (unsigned integer)

The number of results which were returned by the query that caused the search interface to log a search event.

If specified, the numberOfResults value must be the same as the totalCount value in the response body of the corresponding search request to the Search API.

{
  ...
  "numberOfResults": 1866,
  ...
}

originContext (string)

The origin of the event. Used to specify the deployment from which the user performs the action. Suggested values are Search, InternalSearch, or CommunitySearch, or the originLevel1 value.

{
  ...
  "originContext": "CommunitySearch",
  ...
}

originLevel3 (string)

The URL of the page that redirected the browser to the search interface from which the search event originates, i.e., document.referrer.

{
  ...
  "originLevel3": "https://support.coveo.com/s/search/All/Home/%40uri",
  ...
}

queryPipeline (string)

The name of the query pipeline which processed the query that caused the search interface to log a search event.

If specified, the queryPipeline value must be the same as the pipeline value in the request body of the corresponding query to the Search API.

{
  ...
  "queryPipeline": "External Search",
  ...
}

splitTestRunName (string)

The name of the A/B test run, if one is active. See Adding and Managing A/B Tests.

{
  ...
  "splitTestRunName": "splitTest",
  ...
}

splitTestRunVersion (string)

The version of the A/B test run, if one is active.

{
  ...
  "splitTestRunVersion": "splitTestRunAlpha",
  ...
}

userDisplayName (string)

The display name of the user performing the search event. If unspecified and the request is authenticated with a search token, the service attempts to extract the userdisplayName value from the access token that authenticated the log search event request.

{
  ...
  "userDisplayName": "Alice",
  ...
}

userGroups (string array)

The groups that the end user performing the event is a member of. If the request is authenticated with a search token, the service also attempts to extract the userGroups value from the access token that authenticated the log search event request, and logs those groups as well.

{
  ...
  "userGroups": ["Employee"],
  ...
}

Search Event customData (key-value pairs)

Custom data key-value pairs that can contain all the user-defined dimensions and their values, which can be used for reporting or learning.

  • The customData section must be valid JSON.

  • When possible, it is recommended to use a string as the value.

  • The value could be any valid JSON, but it is handled as string over Coveo usage analytics. The string conversion could result in alteration of the original JSON. Keys can only contain alphanumeric or underscore characters. Spaces in the key are replaced with underscores. Uppercase characters in the key are converted to lowercase characters. ‌

  • "key" : "value" → Recommended format for Coveo usage analytics, which will render the string as "value" and make it usable as any other string dimension type over Coveo usage analytics.
  • "key" : [ "value_1", "value_2" ] → Will become the string "[ "value_1", "value_2" ] " within Coveo usage analytics. Although harder to use for reporting, this value could still be useful (e.g., when using filter "contains" type in Coveo usage analytics).
  • It is highly recommended that you create your custom dimension before adding customData (see Adding and Managing Dimensions on Custom Metadata).
  • Any field beginning with context_ will be used by Coveo ML as user context to learn and provide boosts to contextually relevant items.
  • When logging a search event which involved a large query expression (lq) (lq), you can optionally include, as a custom data field, the refinedKeywords (string array) returned by the Search API. These refinedKeywords are extracted from lq by the Coveo ML Intelligent Term Detection (ITD) feature. Although not used by Coveo ML to learn, the refinedKeywords will be available for reporting.
  • Facet toggling
{
  ...
  "customData":{
    "context_usertype" : "Guest",
    "facetId": "@objecttype",
    "facetTitle": "Type",
    "facetValue": "Developers Topic"
  },
  ...
}
  • Case deflection (which involved lq)
{
  ...
  "customData":{
    "context_usertype" : "User",
    "refinedKeywords": ["issue", "making", "request", "403", "api"]
  },
  ...
}