Create and manage an Automatic Relevance Tuning (ART) model
Create and manage an Automatic Relevance Tuning (ART) model
Coveo Machine Learning (Coveo ML) Automatic Relevance Tuning (ART) models provide to end-users the search results that are the most relevant for their queries.
To take advantage of Coveo ML ART, members with the required privileges must first create their ART models.
Prerequisites
Before creating an ART model, you should ensure that your Coveo organization has collected enough usage analytics data. To start providing recommendations, the model needs:
-
At least 100 click and search events [1] (see Sending the required usage analytics event data).
-
To be part of a search interface that records at least 55 visits per day. Each visit must contain at least one user query that has been followed by a click for a specific language.[2]
To analyze whether the search interface in which you want to integrate the ART model produces enough data, see Troubleshoot ART models.
Create an ART model
-
Depending on whether models have already been created in your Coveo organization:
-
If your Coveo organization doesn’t contain any models, on the Models (platform-ca | platform-eu | platform-au) page, click the Automatic Relevance Tuning card.
-
If your Coveo organization already contains models, on the Models (platform-ca | platform-eu | platform-au) page, click Add model, and then click the Automatic Relevance Tuning card.
-
-
Click Next.
-
(Optional) In the Learning interval section, you can change the default and recommended Data period and Building frequency.
-
Click Next.
-
(Optional) Under Include commerce events, activate the Include commerce events option if you want your ART model to automatically consider purchase and cart event data.
Leading practiceIn Coveo for Commerce implementations, such as when configuring an ART model for listing pages, it’s recommended to activate this option. By default, the model only considers the most-clicked products for a given query (from a specific search interface and tab combination). When the option is enabled, the model also considers all "add to cart" and "purchase" events occurring within a particular site (or application) to boost products in search results and product listing pages.
-
(Optional) In the Apply filters on dataset section, you can add filters to refine the data that the model uses to make its recommendations. By narrowing down the dataset that a model uses, you can better customize relevancy for specific user groups and use cases. You can apply filters on all events, or on every event that belongs to a specific category, such as search, click, view, or custom events.
ExampleYour Community search and Agent search have very specific vocabulary. You don’t want them to influence one another in the ART model learning process, so you add a filter on the Origin 1 (Page/Hub) dimension.
This allows your model to learn independently from the actions performed on the different search interfaces.
The Data volume preview section shows the impact of filters on the data that’s available to train the model. This section includes data gathered only during the last seven days and doesn’t consider the selections from the Learning interval section.
-
In the Select a dimension dropdown menu, select the dimension on which you want to base the learning of the model.
-
In the Select an operator dropdown menu, select the appropriate operator.
-
In the Select value(s) dropdown menu, add, type, or select the appropriate value.
-
You can optionally add other filters by clicking Add.
NoteWhen configuring an ART model that doesn’t serve a commerce interface:
-
Don’t configure filters based on Custom dimensions as ART models don’t use custom events in their learning datasets. Doing so would result in the model not applying any filtering.
-
You should verify that the
customEventFilter
field is empty when inspecting your [model’s JSON configuration].
-
-
-
Click Next.
-
In the Name your model input, enter a meaningful display name for the model, and then click Start building.
NoteOn the Models (platform-ca | platform-eu | platform-au) page, under the Status column, in the model row, the value is most probably Inactive.
The model value will change to Active when the model creation is complete (typically within 30 minutes, depending on the amount of usage analytics data to process). The model can only return recommendations when its status is Active.
For more information on Coveo ML model statuses, see the Status column reference.
-
(Optional) If you want to specify advanced parameters for your model, you can use the Advanced tab of the model configuration.
-
You can then associate the model with a pipeline to take advantage of the model in a search interface and test your model to ensure that it behaves as expected.
Make sure to follow the model association leading practices before associating your model with your production query pipeline.
If you have the Enterprise edition, group this ART model and your other implementation resources together in a project. See Manage projects. |
Edit an ART model
-
On the Models (platform-ca | platform-eu | platform-au) page, click the model you want to edit, and then click Edit in the Action bar.
-
On the subpage that opens, select the Configuration tab.
-
Under Name, you can optionally edit the model’s display name.
-
(Optional) Under Include commerce events, you can update the value of the Include commerce events option depending on whether you want your ART model to automatically consider purchase and cart event data.
-
(Optional) In the Learning interval section, you can change the default and recommended Data period and Building frequency.
-
(Optional) In the Apply filters on dataset section, you can add filters to refine the data that the model uses to make its recommendations. By narrowing down the dataset that a model uses, you can better customize relevancy for specific user groups and use cases. You can apply filters on all events, or on every event that belongs to a specific category, such as search, click, view, or custom events.
ExampleYour Community search and Agent search have very specific vocabulary. You don’t want them to influence one another in the ART model learning process, so you add a filter on the Origin 1 (Page/Hub) dimension.
This allows your model to learn independently from the actions performed on the different search interfaces.
The Data volume preview section shows the impact of filters on the data that’s available to train the model. This section includes data gathered only during the last seven days and doesn’t consider the selections from the Learning interval section.
-
In the Select a dimension dropdown menu, select the dimension on which you want to base the learning of the model.
-
In the Select an operator dropdown menu, select the appropriate operator.
-
In the Select value(s) dropdown menu, add, type, or select the appropriate value.
-
You can optionally add other filters by clicking Add.
NoteWhen configuring an ART model that doesn’t serve a commerce interface:
-
Don’t configure filters based on Custom dimensions as ART models don’t use custom events in their learning datasets. Doing so would result in the model not applying any filtering.
-
You should verify that the
customEventFilter
field is empty when inspecting your [model’s JSON configuration].
-
-
-
Click Save.
-
On the Models (platform-ca | platform-eu | platform-au) page, under the Status column, in the model row, the value is most probably Updating.
NoteThe model value will change to Active when the model edition is complete (typically within 30 minutes, depending on the amount of usage analytics data to process). The model can only return recommendations when its status is Active.
For more information on Coveo ML model statuses, see the Status column reference.
-
You can then associate the model with a pipeline to take advantage of the model in a search interface and test your model to ensure that it behaves as expected.
Make sure to follow the model association leading practices before associating your model with your production query pipeline.
Edit an ART model JSON configuration
-
Access the Models (platform-ca | platform-eu | platform-au) page.
-
Click the desired model, and then click More > Edit JSON in the Action bar.
-
In the Edit a Model JSON Configuration panel that appears, modify the existing model configuration:
{ "modelDisplayName": "<MODEL_DISPLAY_NAME>", "exportPeriod": "<EXPORT_PERIOD>", "intervalTime": <INTERVAL_TIME>, "intervalUnit": "<INTERVAL_UNIT>", "commonFilter": "<COMMON_FILTER>", "customEventFilter": "<CUSTOM_EVENT_FILTER>", "exportOffset": "<EXPORT_OFF_SET>", "searchEventFilter": "<SEARCH_EVENT_FILTER>", "viewEventFilter": "<VIEW_EVENT_FILTER>", "extraConfig": [<ADVANCED_ML_PARAMETERS>], }
Where:
-
modelDisplayName
(string) is the name of the model appearing on the Models (platform-ca | platform-eu | platform-au) page. -
exportPeriod
(ISO-8601 string, required) is the period defining the age of the usage analytics data used to build the model. Must be in the ISO8601 period format (that is,PyYmMwWdDThHmMsS
).NoteUnless an
exportOffset
is specified, theexportPeriod
uses the moment when the model was generated as a base. -
intervalTime
(integer, required) is the number ofintervalUnit
(that is,DAY
,WEEK
, orMONTH
) between each update of the model. Must be between1
and30
inclusively. -
intervalUnit
(string enum, required) is the duration unit of the interval between each update of the model. SeeintervalTime
. Accepted values are:DAY
,WEEK
, andMONTH
. -
commonFilter
(string) is the filter to apply to the common event dimensions (shared by all event types) in the export. Multiple filter parameters are joined with theAND
operator. -
customEventFilter
(string) is the filter to apply to the custom event dimensions in the export. Multiple filter parameters are joined with theAND
operator. -
exportOffset
(ISO-8601 string) is the offset of the usage analytics data used to build the model. Must be in the ISO8601 period format (that is,PyYmMwWdDThHmMsS
). The default value isPT0S
, meaning that all events are considered when building a model (theexportPeriod
is based on the moment the model was generated).ExampleYou want to ignore events that occur on the current day, so you set the
exportOffset
value toP1D
. -
searchEventFilter
(string) is the filter to apply to the click and search event dimensions in the export. Multiple filter parameters are joined with theAND
operator. -
viewEventFilter
(string) is the filter to apply to the view event dimensions (shared by all event types) in the export. Multiple filter parameters are joined with theAND
operator. -
extraConfig
(array of string) are additional advanced parameters used to tailor the model to your use case.
-
-
Click Save to apply your changes.
Delete an ART model
You must dissociate a model from all its associated query pipelines before deleting it. Models aren’t automatically dissociated from pipelines when they’re deleted. |
-
On the Models (platform-ca | platform-eu | platform-au) page, click the ML model that you want to delete, and then click More > Delete in the Action bar.
-
In the Delete a Model panel that appears, click Delete model.
Review active model information
On the Models (platform-ca | platform-eu | platform-au) page, click the desired model (must be Active), and then click Open in the Action bar (see Reviewing Coveo Machine Learning model information).
Reference
ART models reference
Default number of recommended results: 5
Note
ART recommends the 5 best learned search results, but may recommend fewer than 5 when:
|
Facts
-
Following an empty query, ART returns the most clicked items during the data period of the model.
-
ART can inject items that wouldn’t normally be included in search results because they were learned to be relevant even if they don’t contain some or all of the searched keywords. This is one of the key ART benefits as a user can find the most useful items without having to type the right keywords or the specific synonym contained these items. This is the default behavior (the Match the Query parameter default value is
false
). -
ART currently ignores all special characters or operators in the user entered query to only keep the keywords and therefore ignores the special behavior described in Using Special Characters in Queries or Search Prefixes and Operators.
-
This ART behavior may lead to unexpected search results when users want to take advantage of more advanced query syntax.
A user is searching for items containing a specific phrase by entering the phrase enclosed in double quotes (see Searching for a phrase) and no items contain the phrase. The user would expect to get no search results, but ART can inject items that were clicked for similar queries made of one or some of the searched keywords (not in a phrase search).
"Status" column
On the Models (platform-ca | platform-eu | platform-au) page of the Administration Console, the Status column indicates the current state of your Coveo ML models.
The following table lists the possible model statuses and their definitions:
Status | Definition | Status icon |
---|---|---|
Active |
The model is active and available. |
|
Build in progress |
The model is currently building. |
|
Inactive |
The model isn’t ready to be queried, such as when a model was recently created or the organization is offline. |
|
Limited |
Build issues exist that may affect model performance. |
|
No query pipeline |
The model isn’t associated with a query pipeline. |
|
No case assist configuration |
The model isn’t associated with a case assist configuration. |
|
Soon to be archived |
The model will soon be archived because it hasn’t been queried for an extended period of time. |
|
Error |
An error prevented the model from being built successfully. |
|
Archived |
The model was archived because it hasn’t been queried for at least 30 days. |
n/a |
“Learning interval” section
In the Learning interval section, you can modify the following:
Set the Coveo ML model training Building frequency based on the Data Period value. Less frequent for a larger Data Period and more frequent for a smaller Data Period as recommended in the following table.
Data period |
Building frequency |
||
---|---|---|---|
Daily |
Weekly |
Monthly |
|
1 month |
|||
3 months (Recommended) |
|||
6 months |
The more data the model has access to and learns from, the better the recommendations. As a general guide, a usage analytics dataset of 10,000 queries or more typically allows a Coveo ML model to provide very relevant recommendations. You can look at your Coveo Usage Analytics (Coveo UA) data to evaluate the volume of queries on your search hub, and ensure that your Coveo ML models are configured with a training Data period that corresponds to at least 10,000 queries. When your search hub serves a very high volume of queries, you can consider reducing the data period so that the model learns only more recent user behavior and be more responsive to trends.
A Coveo ML model regularly retrains on a more recent Coveo UA dataset, as determined by the Building frequency and Data period settings, to ensure that the model remains up-to-date with the most recent user behavior.
Note
If you’re testing the model in a sandbox environment in which very little analytics data is available to train the model, you can activate the Test configuration mode advanced option to ensure the model provides recommendations. |
Required privileges
By default, members with the required privileges can view and edit elements of the Models (platform-ca | platform-eu | platform-au) page.
The following table indicates the privileges required to use elements of the Models page and associated panels (see Manage privileges and Privilege reference).
Action | Service - Domain | Required access level |
---|---|---|
View models |
Machine Learning - Models |
View |
Edit models |
Organization - Organization |
View |
Machine Learning - Models |
Edit |