--- title: Inspect Coveo Machine Learning models slug: mc2g0297 canonical_url: https://docs.coveo.com/en/mc2g0297/ collection: project-guide source_format: adoc --- # Inspect Coveo Machine Learning models [Coveo Machine Learning (Coveo ML)](https://docs.coveo.com/en/188/) [models](https://docs.coveo.com/en/1012/) leverage AI-powered recommendations to provide your users with the best possible search experience according to their context, so ensure that your models are properly implemented and well-trained. > **Important** > > This guide provides guidelines to analyze, review, and test different aspects of a Coveo-powered implementation. > > It shouldn't be considered an official, definitive guide to testing an implementation before it goes live, but rather a guide showcasing the basic components that Coveo implementations typically include. > > Every Coveo implementation is different and may require custom configuration. > > Therefore, you may notice discrepancies between the components and features listed in this guide and those you're using in your actual implementation. > > If you encounter issues, or have any questions related to this article, contact [Coveo Support](https://connect.coveo.com/s/case/Case/Default) for help. ## Access your browser's network monitoring tool . Open your web browser's developer tools. > **Note** > > The examples in this article use the **Google Chrome** developer tools. > For browser-specific information, see: > > * [Google Chrome](https://developers.google.com/web/tools/chrome-devtools/open) > > * [Mozilla Firefox](https://developer.mozilla.org/en-US/docs/Tools) > > * [Safari](https://support.apple.com/en-ca/guide/safari/sfri20948/mac) . Select the **Network** tab. ## Leverage Coveo Machine Learning If your implementation doesn't leverage [Coveo ML](https://docs.coveo.com/en/188/), see [Leverage AI and ML](https://docs.coveo.com/en/3372/) to learn more about Coveo's available machine learning and deep learning algorithms. ## Configure and train Coveo Machine Learning models Depending on your context, needs, and available data, you may have to change the configuration and [training settings](https://docs.coveo.com/en/1727#training-and-retraining) for the default [Coveo ML](https://docs.coveo.com/en/188/) models. ## Review Coveo Machine Learning candidates using the "Models" page You can use the [**Models**](https://platform.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/)) page of the [Coveo Administration Console](https://docs.coveo.com/en/183/) to ensure that your [Coveo ML](https://docs.coveo.com/en/188/) models have [relevant candidates](https://docs.coveo.com/en/1894/) to suggest to your users. > **Important** > > * [Coveo ML](https://docs.coveo.com/en/188/) models leverage [Coveo Analytics](https://docs.coveo.com/en/182/) to provide suggestions, so [ensure that all of the data that needs to be recorded within your search interfaces is properly tracked](https://docs.coveo.com/en/mc2g0166/). > > * Before reviewing the candidates suggested by [Coveo ML](https://docs.coveo.com/en/188/), you should ensure that you have met the prerequisites to [create your desired model](https://docs.coveo.com/en/1832/). > > * You can't currently use the [**Models**](https://platform.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/)) page to inspect the candidates of [Coveo ML](https://docs.coveo.com/en/188/) [Case Classification](https://docs.coveo.com/en/l57e0557/) and [Smart Snippet](https://docs.coveo.com/en/l6eb0531/) models. > However, you can use the [Get detailed information about a specific model](https://platform.cloud.coveo.com/docs?urls.primaryName=Machine%20Learning#/Machine%20Learning%20Models/rest_organizations_paramId_machinelearning_models_paramId_details_get) call of the [Machine Learning Models API](https://docs.coveo.com/en/19/) to get information about your model. Once you have met the prerequisites to create your model, you can review the candidates (if any) that are suggested by [Coveo ML](https://docs.coveo.com/en/188/). To review these candidates and other useful information about your model, access the [**Models**](https://platform.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/ai-and-ml/models/)) page, and then select the model that you want to inspect. You can [use the information provided](https://docs.coveo.com/en/1894#model-information-reference) on this page to ensure that all of the prerequisites for model creation have been met. You can also review [Coveo ML](https://docs.coveo.com/en/188/) candidates that are automatically shown to your users. ## Automatic Relevance Tuning models Typically, [Automatic Relevance Tuning (ART)](https://docs.coveo.com/en/1013/) boosts the ranking score of the five best search results for a given query. ### Test your ART model Once you've analyzed the candidates for your ART model on the **Models** page, you may want to test its workability in your actual implementation. . Access a search interface which leverages the DNE model that you want to test. . [Access your browser's network monitoring tool](#access-your-browsers-network-monitoring-tool). . In the search box, perform a query for which your model [is able to recommend items](https://docs.coveo.com/en/1894/). . Enable debugging mode by appending `&debug=true` to the current URL. ![Enable debugging mode | Coveo](https://docs.coveo.com/en/assets/images/debugging-mode-url.png) . In the network monitoring tool, under the **Name** column, select the latest request to the Search API. The request path should contain `/rest/search/v2`. . Select the **Preview** tab. . Find and expand the `executionReport` property, and then expand the `children` sub-property. You should now be able to see the whole path that your query took before the results were returned to the search interface. ![View query path in executionReport | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/execution-report-children.png) . Expand the `PreprocessQuery` property, and then expand the `children` sub-property. . Expand the `CallingRevealTopClicks` property, and then expand the `response` sub-property. . Expand the `predictions` property. ![ART model predictions | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/art-model-predictions.png) Under `predictions`, you should see the items that were suggested by ART and their [ranking score](https://docs.coveo.com/en/1624/). > **Note** > > Each suggested item shown under `predictions` is identified by the value of its `contentIdValue` [field](https://docs.coveo.com/en/200/). ## Query Suggestion models [query suggestion](https://docs.coveo.com/en/1015/) models recommend relevant queries to users as they type in the search box. For more information on QS models, see: * [Providing Coveo Machine Learning Query Suggestions](https://docs.coveo.com/en/340#providing-coveo-machine-learning-query-suggestions-recommended) * [Create a Query Suggestion model](https://docs.coveo.com/en/3398/) ### Test your QS model Once you've [reviewed query suggestion candidates](https://docs.coveo.com/en/2010/), you may want to test the workability of the QS model in your actual implementation. . [Ensure that Coveo ML query suggestions are logged](https://docs.coveo.com/en/mc2g0129/). . In the search box, start typing queries for which your model [is able to provide suggestions](https://docs.coveo.com/en/1894/) . Ensure that the expected suggestions are displayed. ![Query suggestions in a search box | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/qs-gif.gif) > **Note** > > In [non-production environments](https://docs.coveo.com/en/2959/), you may not have enough data to see query suggestions appear in the search box. > > This is because QS models [have thresholds](https://docs.coveo.com/en/2010/) to prevent infrequent queries from polluting the suggestions. > They only deliver suggestions for queries that were performed and followed by at least one click on search results. > > However, when testing a QS model in a non-production environment, you can [activate the test configuration mode](https://docs.coveo.com/en/3398#test-configuration-mode) to reduce the frequency thresholds which discard queries or clicks that weren't performed frequently enough. ## Content Recommendation models [Coveo ML](https://docs.coveo.com/en/188/) [Content Recommendation (CR)](https://docs.coveo.com/en/1016/) models use views and search actions from all of your users to predict and suggest the most relevant content for the user in their current session. To learn how to configure recommendations, see [Deploy Content Recommendations (CR)](https://docs.coveo.com/en/1886/). ### Test your CR model . Access a search interface which leverages the DNE model that you want to test. . [Access your browser's network monitoring tool](#access-your-browsers-network-monitoring-tool). . Access one of the pages that should be considered as a recommendation by your CR model. . Verify that [a view event is logged](https://docs.coveo.com/en/mc2g0537#test-view-events) when the page loads. . In the network monitoring tool, under the **Name** column, select the view event request. The request path should contain `/rest/ua/v15/analytics/view`. . Select the **Payload** tab. You should now see the request payload sent to the Search API. ![Request payload of view event | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/view-event-request-payload.png) . You should, at a minimum, find the following fields and note their values: -- ** `contentIdKey` ** `contentIdValue` ** `location` -- > **Notes** > > * If you also want this page to be considered a specific type of item, ensure that the `contentType` field is populated as expected. > > * If you leverage custom user context, you should also validate that the `customData` object contains the expected key-values. . Validate that the recommended items match the view event request. .. Access the [**Content Browser**](https://platform.cloud.coveo.com/admin/#/orgid/content/browser/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/content/browser/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/content/browser/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/content/browser/)) page of the Administration Console. .. In the search box, enter the `contentIdKey` and `contentIdValue` that were returned in the view event request payload, separated by the `Contains` operator (`=`). .. The item that you inspected should be the only search result returned in the **Content Browser**. ![Item in Content Browser | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/item-in-content-browser.png) . Access another page that contains a recommendation interface whose output should come from your CR model. . In the network monitoring tool, under the **Name** column, select the latest request to the Search API. The request path should contain `/rest/search/v2`. . Select the **Payload** tab. You should now see the request payload sent to the Search API. . Find and expand the [`actionsHistory`](https://docs.coveo.com/en/13#operation/searchUsingPost-actionsHistory) property, which is essential for content recommendations to work properly. -- Verify that `actionsHistory` meets the following criteria: ** It contains an object that matches a view event for the page from which you accessed the current page. ** Its latest item is an object that matches a view event for the current page. -- ![Recommended item actionsHistory property | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/recommended-item-actionshistory.png) . Ensure that the `recommendation` property contains a value. . Select the **Preview** tab. . Ensure that the request is routed to the [query pipeline](https://docs.coveo.com/en/180/) that contains the CR model. ![Recommended item pipeline | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/recommended-item-pipeline.png) ## Dynamic Navigation Experience models If you leverage the [Coveo ML](https://docs.coveo.com/en/188/) [Dynamic Navigation Experience (DNE)](https://docs.coveo.com/en/2907/), you may want to ensure that the model orders facets and facet values according to the current context (see [Deploy Dynamic Navigation Experience (DNE)](https://docs.coveo.com/en/2918/)). First, ensure that the search interface incorporates the required components: * If your interface relies on the [Coveo Atomic library](https://docs.coveo.com/en/lcdf0264/): ** [`atomic-facet`](https://static.cloud.coveo.com/atomic/v3/storybook/index.html?path=/docs/atomic-facet\--docs) ** [`atomic-facet-manager`](https://static.cloud.coveo.com/atomic/v3/storybook/index.html?path=/docs/atomic-facet-manager\--docs) * If your interface relies on the [Coveo Headless library](https://docs.coveo.com/en/lcdf0493/): ** [`Facet`](https://docs.coveo.com/en/headless/latest/reference/interfaces/Search.Facet.html) ** [`FacetManager`](https://docs.coveo.com/en/headless/latest/reference/interfaces/Search.FacetManager.html) ### Test your DNE model . Access a search interface which leverages the DNE model that you want to test. . [Access your browser's network monitoring tool](#access-your-browsers-network-monitoring-tool). . In the search box, perform a query for which your model is [able to recommend items](https://docs.coveo.com/en/1894/). . Enable debugging mode by appending `&debug=true` to the current URL. ![Enable debugging mode | Coveo](https://docs.coveo.com/en/assets/images/debugging-mode-url.png) . In the network monitoring tool, under the **Name** column, select the latest request to the Search API. The request path should contain `/rest/search/v2`. . Select the **Preview** tab. . Find and expand the `executionReport` property, and then expand the `children` sub-property. You should now be able to see the whole path that your query took before the results were returned to the search interface. ![View query path in executionReport | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/execution-report-children.png) . Expand the `PreprocessQuery` property, and then expand the `children` sub-property. . Expand the `CallingRevealFacetSense` property, and then expand the `response` sub-property. . Expand the `facetSenseResults` property. Under `facetSenseResults`, you should see several properties which reveal actions from DNE. These include the following: -- ** `facetOrdering`: the order in which the facetable fields are presented to the user according to their context. ** `rankingBoost`: the boost granted to the facetable fields according to the user's context. ** `valuesOrdering`: the order in which the values for each facet are displayed. -- ![DNE facetSenseResults property | Coveo](https://docs.coveo.com/en/assets/images/coveo-solutions/dne-facetsenseresults.png) . Find the `userContext` property. This displays the context upon which DNE bases its suggestions, apart from the query itself. ## Smart Snippet models If you leverage [Coveo ML](https://docs.coveo.com/en/188/) [Smart Snippets](https://docs.coveo.com/en/laea5490/), you may want to ensure that your model properly provides snippets of text. First, ensure that the search interface incorporates the required components: * If your interface relies on the [Atomic library](https://docs.coveo.com/en/atomic/latest/): ** [`atomic-smart-snippet`](https://static.cloud.coveo.com/atomic/v3/storybook/index.html?path=/docs/atomic-smart-snippet\--docs) ** [`atomic-smart-snippet-suggestions`](https://static.cloud.coveo.com/atomic/v3/storybook/index.html?path=/docs/atomic-smart-snippet-suggestions\--docs) * If your interface relies on the [Headless library](https://docs.coveo.com/en/headless/latest/): ** [`SmartSnippet`](https://docs.coveo.com/en/headless/latest/reference/interfaces/Search.SmartSnippet.html) ** [`SmartSnippetQuestionsList`](https://docs.coveo.com/en/headless/latest/reference/interfaces/Search.SmartSnippetQuestionsList.html) ### Test your Smart Snippet model You can test the model on a search interface that contains the required components, and for which the traffic is directed to the [query pipeline that you associated with your Coveo ML Smart Snippet model](https://docs.coveo.com/en/l9le0392#step-4-associate-the-model-with-the-desired-query-pipeline). Perform a query that would likely trigger a snippet to appear in the search results. **Example** You scope an item for your model from which to extract snippets. It contains a header that reads `When exactly is a model retrained?`. When you perform the query `When is a model retrained`, you see the following: ![Example of a Smart Snippet model in action | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/ss-example-gif.gif) You can also inspect the request to the Search API to get detailed information about the model's output for a given query. . Access a [search interface which contains the required components](https://docs.coveo.com/en/l8ve0354/), and in which the traffic is directed to the query pipeline that you [associated with your Coveo ML Smart Snippet model](https://docs.coveo.com/en/l6he0309/). . [Access your browser's network monitoring tool](https://docs.coveo.com/en/mc2g0297#access-your-browsers-network-monitoring-tool). . In the search box, perform a query that would likely trigger a snippet to appear in the search results. . In the network monitoring tool, under the **Name** column, select the latest request to the Search API. The request path should contain `/rest/search/v2`. . Select the **Preview** tab. . Find and expand the [`questionAnswer`](https://docs.coveo.com/en/l9le0392#search-api-response-reference) property. It provides detailed information about the model's output for this specific query. This property won't appear if the model can't provide snippets for the current query. ![Search API response for a smart snippet model | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/ss-questionanswer.png)