---
title: Troubleshoot Automatic Relevance Tuning models
slug: '1858'
canonical_url: https://docs.coveo.com/en/1858/
collection: leverage-machine-learning
source_format: adoc
---
# Troubleshoot Automatic Relevance Tuning models
[Coveo Machine Learning (Coveo ML)](https://docs.coveo.com/en/188/) [Automatic Relevance Tuning (ART)](https://docs.coveo.com/en/1013/) [models](https://docs.coveo.com/en/1012/) learn from click and search [events](https://docs.coveo.com/en/260/) that occurred sequentially during the same [visit](https://docs.coveo.com/en/271/).
Based on those events, a given ART model extracts candidates, which are [queries](https://docs.coveo.com/en/231/) for which the model can recommend at least one [item](https://docs.coveo.com/en/210/), and provides the most relevant items as top search results.

> **Notes**
>
> * An ART model learns from the way users interact with your website.
> The more traffic you have, the better the model will get.
> However, an ART model needs a minimum of 100 click and search events to start boosting items among the top search results (see [About search result ranking](https://docs.coveo.com/en/1624/)).
>  
> With under 100 events, an ART model doesn't boost the ranking weight of any item.
> You can however [reduce the amount of events required to build the model](https://docs.coveo.com/en/2935/) if needed.
>  
> To help a new model, you can [train it by linking queries to results](#training-art-models-by-linking-queries-to-results).
> 
> * In Coveo for Commerce implementations, ART can also learn from [purchase](https://docs.coveo.com/en/l39m0327/) and [cart](https://docs.coveo.com/en/n39h1594/) events.
> When [creating an ART model](https://docs.coveo.com/en/3397/), you can activate the **Include commerce events** option for the model to consider these events.

> **Important**
>
> ART requires real data from a specific community to ensure ART results respond to real user intentions.
> Therefore, it's strongly NOT recommended to provide false generated data.

## Sending the required usage analytics event data

Items can be used in ART model building only if they're identifiable from the Coveo Analytics events that pertain to them.
To that end, the [`contentIdKey`](https://docs.coveo.com/en/2064#contentidkey-string) and [`contentIdValue`](https://docs.coveo.com/en/2064#contentidvalue-string) parameters must be present in the [`customData`](https://docs.coveo.com/en/2064#click-event-customdata-key-value-pairs) of the click events on that item.

For technical details on those events, see [Log click events](https://docs.coveo.com/en/2064/).

## Reviewing your Coveo ML ART model learning dataset

When you have the [required privileges](https://docs.coveo.com/en/1964#required-privileges), you can use the **Analytics** section of the [Coveo Administration Console](https://docs.coveo.com/en/183/) to browse user visits, and therefore evaluate if your search interface produces enough [data](https://docs.coveo.com/en/259/).

The following procedure assumes that you're familiar with [global dimension filters](https://docs.coveo.com/en/1675/) and Visit Browser features (see [Review user visits with the Visit Browser](https://docs.coveo.com/en/1964/)).

. On the [**Visit Browser**](https://platform.cloud.coveo.com/admin/#/orgid/usage/visit/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/usage/visit/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/usage/visit/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/usage/visit/)) page:

![Visit Browser interface showing search event filters | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/visit-browser-art.png)

 .. Select a date interval of three months (see [Review search usage data by date interval](https://docs.coveo.com/en/1964#review-search-usage-data-by-date-interval)).

 .. In the **Show visits containing** section, add the following filters (see [Add visit filters](https://docs.coveo.com/en/1964#add-visit-filters)):

  *** **a search event** WHERE:

   **** Query is not blank or n/a

   **** Origin 1 (page/hub) is [`Search page or search hub name`]

   **** Origin 2 (tab/interface) is [`Tab or search interface name`]

   **** Language is [`Language`]

  *** and at least **a click event**

. At the lower right of the screen, you can see the number of visits in the selected period from which an ART model could learn.

![Location and browser information with operating system details | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/admin-visit-browser-art.png)

## Training ART models by linking queries to results

When leveraging Coveo ML, manually adjusting the relevance of search results (for example, creating [query pipeline](https://docs.coveo.com/en/180/) rules) becomes less necessary.
For example, you no longer have to create thesaurus rules since ART models recommend the same results for different queries when they contain synonyms simply by analyzing the [data](https://docs.coveo.com/en/259/) of a specific [search interface](https://docs.coveo.com/en/2741/).
You can help accelerate the learning process of an ART model by linking queries to results such as pointing synonyms as well as words behind acronyms.

> **Notes**
>
> * The help you provide lasts as long as the data is used to train the model in the worst-case scenario (see [Data period](https://docs.coveo.com/en/3397#learning-interval-section)).
> Best-case scenario, your search interface [users](https://docs.coveo.com/en/250/) repeat the actions you take to train the model.
>  
> For example, you point to the ART model that `BO` means `business optimization` and your model is based on three months of data.
> In 90 days, you may have to help the model understand the same thing if no users searching for `BO` and `business optimization` clicked the same results.
> 
> * ART models don't know synonyms, but learn links from queries to clicks.
> Therefore, in the example above, an ART model would have learned the link between `BO`, `business optimization`, and the clicked results;
> therefore removing the need for a thesaurus rule.

To help train Coveo ML ART models by linking queries to results

. Ensure that Coveo ML ART is configured and enabled on your search interface (see [Create an Automatic Relevance Tuning model](https://docs.coveo.com/en/3397/)).

. On your search interface, perform a query, and then click the result you want to be recommended.

. Repeat the procedure for each desired query.

. Once the model is trained, on the [**Content Browser**](https://platform.cloud.coveo.com/admin/#/orgid/content/browser/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/content/browser/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/content/browser/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/content/browser/)) page, ensure that the model returns the expected result (see [Inspect items with the Content Browser](https://docs.coveo.com/en/2053/)).
Repeat the procedure if needed.

. Remove any [thesaurus rules](https://docs.coveo.com/en/3405/) once the [data period](https://docs.coveo.com/en/3397#learning-interval-section) for training your ART model has expired.
This is recommended as not doing so can cause the wrong results to be returned in response to users' queries, as demonstrated by the example below:


**Example**

During a shopping session, a customer typically browses through multiple items after performing a query.

Taking advantage of this, you might want to promote a related item that wasn't searched for in the user's query.
An instance of this could be setting up a thesaurus rule which includes search results for `sports socks` every time a user searches for `running shoes`.

Since ART learns from searches and clicks to boost search results, the model will establish a relation between the query `running shoes` and index items representing `sports socks` that were clicked after the user searched for `running shoes`.

After your ART model has been trained, remove this thesaurus rule.
Not doing so will result in including all the index items representing `sports socks` in your search results when the user queries for `running shoes`.
This isn't desirable as only those items representing `sports socks` **that were clicked on** after searching for `running shoes` should be included in the search results.


> **Notes**
>
> * After the next (scheduled) model update, your model will recommend results based on your training.
> 
> * The search results for queries that include the terms you created the rule for will remain the same even after you remove the thesaurus rules (unless the **Match the query** option was selected when [associating the model](https://docs.coveo.com/en/l1ca1038#associate-an-art-model)).
> To ensure that this is the case, you can test your ART model by comparing results when it's associated with a query pipeline that contains the thesaurus rules with a query pipeline that doesn't (see [Manage A/B tests](https://docs.coveo.com/en/3255/)).

## Inspecting search results

You can use the [**Relevance Inspector**](https://platform.cloud.coveo.com/admin/#/orgid/search/relevanceInspector/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/search/relevanceInspector/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/search/relevanceInspector/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/search/relevanceInspector/)) to see the items that were boosted by ART for a given user query.

. [Run the Relevance Inspector](https://docs.coveo.com/en/mbad0273#use-the-relevance-inspector).
. Under **Query pipeline rules and models**, select **Automatic Relevance Tuning**.