Leveraging Custom Contexts in Coveo Machine Learning Features

The Coveo™ Machine Learning (Coveo ML) models learn from user behavior to automatically improve Coveo-powered content relevance (see Coveo Machine Learning Features).

You can take the Coveo ML feature relevance one step further by defining and using custom contexts that are appropriate to your business.

The Coveo ML models favor (or boost the ranking score) of content matching or accessed more often in the specified custom context. Coveo ML models do not filter content based on the custom context, allowing to still provide output even when there are no or little context matches.

Consider specifying a custom context for Coveo ML models when most queries should return different results depending on the context.

In the following case, the user role could be a good custom context.

On your product documentation site, users are authenticated and from their profile, you can get their role relative to your products (end-user, administrator, or developer).

Often, for similar queries such as entering only a feature name, users with different roles have different intentions and look for different kind of information:

  • Developers - look for how to customize the product with code samples.

  • Administrators - look for installation and configuration topics.

  • End-users - look for instructions on how to use the product.

With the user role as a custom context, Coveo ML features can promote content that is more relevant to each user’s context.

You can also use custom context in Coveo Cloud query pipeline conditions and then apply such conditions to query pipeline elements (see Leveraging Query Pipeline Custom Context Information in Salesforce).

How Contexts and Coveo Machine Learning Models Work

In practice, custom contexts are key/value pairs that must be sent as custom data with standard or custom usage analytics events to build a contextualized activity history.

When a user makes a query and a context is specified, matching relevant results with highest activity history in this context are promoted. More technically, the Coveo ML models use the context like an intelligent query ranking expression (QRE), boosting results that match the query and the context. Coveo ML models automatically take into account custom usage analytics metadata keys that start with the context_ prefix.

Other relevant results, not seen in a specified context are not promoted by Coveo ML models, but their overall ranking score may still be higher than the contextually promoted ones when they are really popular.

A query can carry more than one context key/value pairs, in which case the ranking score boosts associated with each key/value context pair are combined.

Custom Context Leveraging Leading Practices

Consider the following criteria for a good custom context:

  • For the same queries, successful results are significantly different depending on the context key/value pairs.

    The context can be the user role. The authenticated users on your software product documentation site play one of the following role relative to your product:

    • End-user - Actually use the product.

    • Administrator - Install, configure, and support the product.

    • Developer - Write code to customize or interact with your product.

    Because users are authenticated, at query time you have a method to get the user role

  • You can clearly identify and send the appropriate static or dynamic key/value pair with each query (see Adding Pipeline Context to a Search Page Using the Interface Editor).

  • While contexts can be combined, use only the few most significant context keys, typically less than 5 gives better results.

    • On your client community, you can send the following contexts since they are different for each client:

      • User role

      • User department

      • Product categories they already own

      • User country (useful when items are different from one country to the other)

    • You have a large number of employees using your agent panel. Therefore, you want to pass information about the role, department, and level of your agents to categorize or group them.

      When logging usage analytics events, you thus ensure that your developer include, in the customData property of the request body, key-value pairs such as:

      • "context_role":"<context_role>"

      • "context_department":"<context_department">

      • "<context_level>":"<context_level">

      where:

      • <context_role> is the role of the agent sending the request.

      • <context_department> is the department of the agent sending the request.

      • <context_level> is the level of the agent sending the request.

    Increasing the number of context combinations dilutes the context-specific effect. Reduce the number of key/value pairs even more when the level of search activity is low.

    Your Coveo-powered customer portal search page serves on average of about 500 000 search events per day, providing plenty of activity from which Coveo ML models can learn for a few contexts, each with a few values.

    On the other hand, your Coveo-powered employees portal search page rather serves less than 1000 search events per day (normal, only a couple hundred employees can access it). With much less data to learn from, Coveo ML models may be effective only for one significant context key with its few values.

  • Each context key should have a limited number of possible values.

    Your product popularity vary notably from one country to the other, so you guess using the search page user country of origin as a context will help Coveo ML models return more relevant content for customers of each country.

    The effect of the context will probably be notable if your customers are from only a few countries, but may be diluted if they come from 100 countries.

  • DO NOT use the following aspects as custom context keys because they are passed by the Coveo JavaScript search interfaces with each query and are taken into account out-of-the-box by Coveo ML models:

    • Language - The language of the search interface.

    • Search Hub - The name you give to a specific search instance.

    • Interface (Tab) - The different search page tabs in your search hub.

    • Is Guest

    • Is Anonymous

    Coveo powers the search pages in your English-only employee portal as well as in your multi-language customer portal.

    In the Spanish customer portal, results automatic relevance tuning, query suggestions, or event recommendations returned out-of-the-box by the Coveo ML models take what other customers searched for in the Spanish portal, not searches made in other languages and not searches made on the employee portal.

Leveraging Custom Context Steps

  1. Define the context key/value pairs that you plan to leverage.

    It is important to pick key/value custom context pairs for which you can get appropriate values in your code and provide good search result differentiation (see Custom Context Leveraging Leading Practices).

  2. Ensure that appropriate context key/value pairs that you defined are passed by your search interface with each query:

  3. Create/edit Coveo ML models

    Existing or new Coveo ML models automatically take advantage of custom context values passed as described in previous steps when the are trained or retrained (see Training and Retraining), so there is no custom context specific configuration to do on Coveo ML models.

    If not already done, create Coveo ML models:

  4. Attempt to measure the impact of custom contexts

    There is no direct methods to measure the impact of the custom contexts on the Coveo ML models output. The best you can do is to create analytics reports to look at some quality metrics (such as Click-Through or Click Rank) as a function of context dimensions.

    1. Create a dimension for each of your custom context key (see Managing Dimensions on Custom Metadata).

      When some custom context key/value pairs have been passed by your search interface, in the Add a Dimension panel, the API name parameter automatically proposes custom context keys (the key is prefixed with context_).

      When your custom context key is userRole, the proposed API name value is context_userRole.

    2. Validate that your Coveo ML model has been trained since you started to send custom context (see Training and Retraining).

    3. Create a report with the metrics of your choice filtered by your context dimension(s) (see Creating Usage Analytics Dashboards).