Inspect Coveo Machine Learning models

This is for:

Developer System Administrator

Coveo Machine Learning (Coveo ML) models leverage AI-powered recommendations to provide your users with the best possible search experience according to their context, so you must ensure that your models are properly implemented and well-trained.

Important

This guide provides guidelines to analyze, review, and test different aspects of a Coveo-powered implementation.

It shouldn’t be considered an official, definitive guide to testing an implementation before it goes live, but rather a guide showcasing the basic components that Coveo implementations typically include.

Every Coveo implementation is different and may require custom configuration.

Therefore, you may notice discrepancies between the components and features listed in this guide and those you’re using in your actual implementation.

If you encounter issues, or have any questions related to this article, contact Coveo Support for help.

Access your browser’s network monitoring tool

  1. Open your web browser’s developer tools.

    Note

    The examples in this article use the Google Chrome developer tools. For browser-specific information, see:

  2. Select the Network tab.

Leverage Coveo Machine Learning

If your implementation doesn’t leverage Coveo ML, see Leverage Machine Learning to learn more about Coveo’s available machine learning and deep learning algorithms.

Configure and train Coveo Machine Learning models

Depending on your context, needs, and available data, you may have to change the configuration and training settings for the default Coveo ML models.

Review Coveo Machine Learning candidates using the "Models" page

You can use the Models (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console to ensure that your Coveo ML models have relevant candidates to suggest to your users.

Important

Once you have met the prerequisites to create your model, you can review the candidates (if any) that are suggested by Coveo ML.

To review these candidates and other useful information about your model, access the Models (platform-ca | platform-eu | platform-au) page, and then select the model that you want to inspect.

You can use the information provided on this page to ensure that all of the prerequisites for model creation have been met. You can also review Coveo ML candidates that are automatically shown to your users.

Automatic Relevance Tuning models

Typically, Automatic Relevance Tuning (ART) boosts the ranking score of the five best search results for a given query.

You can use the Model Testing (platform-ca | platform-eu | platform-au) page of the Administration Console to compare the ranking of two Coveo ML ART models. You can also use it to evaluate the efficiency of a given ART model compared to the default ranking behavior provided without ART.

Test your ART model

Once you have analyzed the candidates for your ART model on the Models page and tested its ranking behavior on the Model Testing page, you may want to test its workability in your actual implementation.

  1. Access a search interface which leverages the ART model that you want to test.

  2. Access your browser’s network monitoring tool.

  3. In the search box, perform a query for which your model is able to recommend items.

  4. Enable debugging mode by appending &debug=true to the current URL.

    Enable debugging mode | Coveo

  5. In the network monitoring tool, under the Name column, select the latest request to the Search API. The request path should contain /rest/search/v2.

  6. Select the Preview tab.

  7. Find and expand the executionReport property, and then expand the children sub-property. You should now be able to see the whole path that your query took before the results were returned to the search interface.

    View query path in executionReport | Coveo

  8. Expand the PreprocessQuery property, and then expand the children sub-property.

  9. Expand the CallingRevealTopClicks property, and then expand the response sub-property.

  10. Expand the predictions property.

    ART model predictions | Coveo

    Under predictions, you should see the items that were suggested by ART and their ranking score.

    Note

    Each suggested item shown under predictions is identified by the value of its contentIdValue field.

Query Suggestion models

You can use the Model Testing (platform-ca | platform-eu | platform-au) page of the Administration Console to compare the suggestions provided by two Coveo ML Query Suggestion (QS) models.

For more information on QS models, see:

Test your QS model

Once you have tested a QS model on the Model Testing page, you may want to test its workability in your actual implementation.

  1. Ensure that Coveo ML query suggestions are logged.

  2. In the search box, start typing queries for which your model is able to provide suggestions

  3. Ensure that the expected suggestions are displayed.

    Query suggestions in a search box | Coveo
Note

In non-production environments, you may not have enough data to see query suggestions appear in the search box.

This is because QS models have thresholds to prevent infrequent queries from polluting the suggestions. They only deliver suggestions for queries that were performed and followed by at least one click on search results.

However, when testing a QS model in a non-production environment, you can activate the test configuration mode to reduce the frequency thresholds which discard queries or clicks that were not performed frequently enough.

Content Recommendation models

Coveo ML Content Recommendation (CR) models use views and search actions from all of your users to predict and suggest the most relevant content for the user in their current session.

To learn how to configure recommendations, see Deploy Content Recommendations (CR).

Test your CR model

  1. Access a recommendation interface which leverages the CR model that you want to test.

  2. Access your browser’s network monitoring tool.

  3. Access one of the pages that should be considered as a recommendation by your CR model.

  4. Verify that a view event is logged when the page loads.

  5. In the network monitoring tool, under the Name column, select the view event request. The request path should contain /rest/ua/v15/analytics/view.

  6. Select the Payload tab. You should now see the request payload sent to the Search API.

    Request payload of view event | Coveo

  7. You should, at a minimum, find the following fields and note their values:

    • contentIdKey

    • contentIdValue

    • location

    Notes

    • If you also want this page to be considered a specific type of item, ensure that the contentType field is populated as expected.

    • If you leverage custom user context, you should also validate that the customData object contains the expected key-values.

  8. Validate that the recommended items match the view event request.

    1. Access the Content Browser (platform-ca | platform-eu | platform-au) page of the Administration Console.

    2. In the search box, enter the contentIdKey and contentIdValue that were returned in the view event request payload, separated by the Contains operator (=).

    3. The item that you inspected should be the only search result returned in the Content Browser.

      Item in Content Browser | Coveo

  9. Access another page that contains a recommendation interface whose output should come from your CR model.

  10. In the network monitoring tool, under the Name column, select the latest request to the Search API. The request path should contain /rest/search/v2.

  11. Select the Payload tab. You should now see the request payload sent to the Search API.

  12. Find and expand the actionsHistory property, which is essential for content recommendations to work properly.

    Verify that actionsHistory meets the following criteria:

    • It contains an object that matches a view event for the page from which you accessed the current page.

    • Its latest item is an object that matches a view event for the current page.

    Recommended item actionsHistory property | Coveo

  13. Ensure that the recommendation property contains a value.

  14. Select the Preview tab.

  15. Ensure that the request is routed to the query pipeline that contains the CR model.

    Recommended item pipeline | Coveo

Dynamic Navigation Experience models

If you leverage the Coveo ML Dynamic Navigation Experience (DNE), you may want to ensure that the model orders facets and facet values according to the current context (see Deploy Dynamic Navigation Experience (DNE)).

You must first ensure that the search interface incorporates the required components:

Test your DNE model

  1. Access a search interface which leverages the DNE model that you want to test.

  2. Access your browser’s network monitoring tool.

  3. In the search box, perform a query for which your model is able to recommend items.

  4. Enable debugging mode by appending &debug=true to the current URL.

    Enable debugging mode | Coveo

  5. In the network monitoring tool, under the Name column, select the latest request to the Search API. The request path should contain /rest/search/v2.

  6. Select the Preview tab.

  7. Find and expand the executionReport property, and then expand the children sub-property. You should now be able to see the whole path that your query took before the results were returned to the search interface.

    View query path in executionReport | Coveo

  8. Expand the PreprocessQuery property, and then expand the children sub-property.

  9. Expand the CallingRevealFacetSense property, and then expand the response sub-property.

  10. Expand the facetSenseResults property.

    Under facetSenseResults, you should see several properties which reveal actions from DNE. These include the following:

    • facetOrdering: the order in which the facetable fields are presented to the user according to their context.

    • rankingBoost: the boost granted to the facetable fields according to the user’s context.

    • valuesOrdering: the order in which the values for each facet are displayed.

    DNE facetSenseResults property | Coveo

  11. Find the userContext property. This displays the context upon which DNE bases its suggestions, apart from the query itself.

Smart Snippet models

If you leverage Coveo ML Smart Snippets, you may want to ensure that your model properly provides snippets of text.

You must first ensure that the search interface incorporates the required components:

Test your Smart Snippet model

You can test the model on a search interface that contains the required components, and for which the traffic is directed to the query pipeline that you associated with your Coveo ML Smart Snippet model.

Perform a query that would likely trigger a snippet to appear in the search results.

Example

You scope an item for your model from which to extract snippets. It contains a header that reads When exactly is a model retrained?.

When you perform the query When is a model retrained, you see the following:

Example of a smart snippet model in action | Coveo

You can also inspect the request to the Search API to get detailed information about the model’s output for a given query.

  1. Access a search interface which contains the required components, and in which the traffic is directed to the query pipeline that you associated with your Coveo ML Smart Snippet model.

  2. Access your browser’s network monitoring tool.

  3. In the search box, perform a query that would likely trigger a snippet to appear in the search results.

  4. In the network monitoring tool, under the Name column, select the latest request to the Search API. The request path should contain /rest/search/v2.

  5. Select the Preview tab.

  6. Find and expand the questionAnswer property. It provides detailed information about the model’s output for this specific query. This property won’t appear if the model can’t provide snippets for the current query.

    Search API response for a smart snippet model | Coveo