Associate an Automatic Relevance Tuning model with a query pipeline

This is for:

Developer System Administrator

When a Coveo Machine Learning (Coveo ML) model has been created, it must be associated with a query pipeline to be effective in a search interface.

organization members with the required privileges can access the Machine Learning tab of a query pipeline configuration page to manage Coveo ML model associations for that query pipeline.

Leading practices

Test the model efficiency

Once you’ve created a Coveo ML ART model, the leading practice is to test the model performance by doing an A/B test.

This allows you to test the model on a chosen proportion of the traffic passing through a given query pipeline. You can then assess the impact of the model by comparing the query pipeline search performance metrics with and without the model.

Once satisfied with the model efficiency, you can stop the A/B test to make the model effective for all the traffic passing through the query pipeline.

Note

You can also use the {model-testing-link} feature of the Coveo Administration Console to compare two ART models together, or to test the results a pipeline-model combination would provide if associated together.

Validate that the model is effective

To validate that your Coveo ML models work as expected, you can inspect your models.

Plan the usage of custom contexts

While Coveo ML models can perform well without custom context information, using custom contexts can take Coveo ML relevance one step further.

You can define custom contexts and then pass appropriate ones along with usage analytics events and queries to allow Coveo ML to take them into account.

Notes

  • Custom context support is limited under the new Event Protocol.

  • If you’re just getting started with Coveo ML, you can skip this step to take advantage of Coveo ML model types more quickly and easily, and consider using custom contexts in a second phase.

Associate an ART model

Important

Follow the model association leading practices when associating your model with your query pipeline.

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline for which you want to associate the model, and then click Edit components in the Action bar.

  2. On the subpage that opens, select the Machine Learning tab, and then in the upper-right corner, click Associate Model.

  3. In the Model dropdown menu, select the desired model.

  4. On the right side, under Condition, you can select a query pipeline condition in the dropdown menu or create a new one.

  5. In the Advanced Configuration section, you can optionally select or deselect the following parameters:

    Important

    For commerce use cases, a specific configuration is required when associating an ART model with a query pipeline. See About ART for Coveo for Commerce for details.

  6. Click Associate Model.

Edit an ART model association

Important

Follow the model association leading practices when associating your model with your query pipeline.

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline for which you want to edit a model association, and then click Edit components in the Action bar.

  2. On the subpage that opens, select the Machine Learning tab, click the desired model, and then click Edit in the Action bar.

  3. On the right side, under Condition, you can select a query pipeline condition in the dropdown menu or create a new one.

  4. In the Advanced Configuration section, you can optionally select or deselect the following parameters:

    Important

    For commerce use cases, a specific configuration is required when associating an ART model with a query pipeline. See About ART for Coveo for Commerce for details.

  5. Click Save.

Associate an ART model via a JSON configuration

Advanced users may want to manage a model association via a JSON configuration to specify association parameters that don’t fit with the parameters available in the Administration Console.

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline for which you want to associate a model, and then click Edit components in the Action bar.

  2. On the subpage that opens, select the Machine Learning tab, and then in the upper-right corner, click dots, and select Associate a model in JSON view.

  3. On the Associate a Model subpage, in JSON view, replace the <Model_ID> placeholder with the actual ID of the model you want to associate with the pipeline (see Review model information).

    Note

    Once you have accessed the Associate a Model subpage in JSON view:

    • You can always go back to the Associate a Model subpage in the UI view and use the available options. However, all unsaved changes made on the Associate a Model subpage in JSON view will be lost.

    • The Associate a Model subpage in JSON view becomes the default model association view for that model. In other words, the Associate a Model subpage in JSON view is now automatically displayed when you access this model association.

  4. Click Associate Model.

Tip
Leading practice

While you can specify model association parameters when creating a model association via a JSON configuration, the leading practice is to first create the association, and then edit its configuration if needed. This allows you to easily tweak the entire configuration of the model association instead of creating it from scratch.

Edit an ART model association via a JSON configuration

Advanced users may want to manage a model association via a JSON configuration to specify association parameters that don’t fit with the parameters available in the Administration Console.

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline for which you want to edit a model association, and then click Edit components in the Action bar.

  2. In the Machine Learning tab, click the desired model, and then click Edit in the Action bar.

  3. On the Edit a Model Association subpage, click dots, and then select Switch to JSON view.

  4. On the Switch to JSON view? panel that appears, click Switch to JSON view.

    Important

    Switching to the JSON view of the Edit a Model Association subpage cancels unsaved configuration changes made on the Model Association page.

  5. On the Edit a Model Association subpage, in JSON view, tune the JSON model association configuration as needed (see Model association parameters).

    Note

    Once you have accessed the Associate a Model subpage in JSON view:

    • You can always go back to the Edit a Model Association subpage in the UI view and use the available options. However, all unsaved changes made on the Associate a Model subpage in JSON view will be lost.

    • If you specified non-default parameters in JSON view of the Edit a Model Association subpage, the complete configuration will be reset to the default one when switching back to the UI view of the Edit a Model Association subpage.

    • The Edit a Model Association subpage in JSON view becomes the default model association view for that model. In other words, the Edit a Model Association subpage in JSON view is now automatically displayed when you access this model association.

  6. Click Save.

Dissociate a model

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline from which you want to dissociate a model, and then click Edit components in the Action bar.

  2. On the subpage that opens, select the Machine Learning tab.

  3. In the Machine Learning tab, click the model you want to dissociate from the pipeline, and then click Dissociate in the Action bar.

Reorder model associations

The Coveo ML models of a given type are executed in the order in which they appear on the page until a condition is satisfied.

Important

The first model on the list will be used if no conditions are met.

  1. On the Query Pipelines (platform-ca | platform-eu | platform-au) page, click the query pipeline in which you want to reorder model associations, and then click Edit components in the Action bar.

  2. On the subpage that opens, select the Machine Learning tab.

  3. In the Machine Learning tab of the desired query pipeline, click the model whose position you want to change, and then use the Move up or Move down arrows in the Action bar to change the position of the model.

Reference

ART advanced configuration options

Ranking modifier

The Recommended value of 250 typically modifies the ranking score of the ART results so that they appear among the top 10 search results.

You should only consider reducing the default value when ART result ranking scores significantly exceed the default ranking, such as when ART results are always appearing among the top five results. You generally don’t want ART results to completely override the normal search results ranking, but rather bring them among the first top 10 results. Typical ranking modifiers are between 50 and 250.

Oppositely, you should only consider increasing the default value when the ART results generally don’t appear within the first search result page. Increase the value until the ART results surface among the top 10 results.

Example

Many customers on a community search page start seeking help on a popular product by entering the part number. However, many useful items available about this product don’t include that part number. Consequently, the initial search results don’t return relevant items, but customers keep adapting their keywords or navigating in search results until they find an appropriate result for the product.

Because ART learns that this result was useful in relation with this part number, when the Match the query checkbox is cleared, ART can now recommend it immediately, even if the result doesn’t match the part number.

Match the advanced query

The Match the advanced query checkbox is selected by default because it’s typically desirable for ART results to match search interface scope, facet selections, and filters. Items recommended by ART are typically not distinguishable from other results in search interfaces. If ART is allowed to inject results outside of the expected scope, the overall search experience may feel confusing for your users.

Example

On a community search page, an end user selects the Knowledge Base Article value in the Item Type facet, enters acme television set in the search box, and then submits the query.

In the query pipeline, the query is processed by an ART model whose Match the advanced query option is enabled. Consequently, the model can only recommend knowledge base articles (that is, items whose @documenttype field value is equal to kbarticle).

If the option was disabled, the model could also recommend items of a different type, such as videos and user manuals. However, the end user likely expects such items to be filtered out, resulting in a confusing experience.

Important

Clearing the Match the advanced query checkbox could lead to security issues if the scope of the search interface that’s served by the ART model is limited by a filter rule that’s based on the aq query parameter.

Therefore, you should only consider clearing the Match the advanced query checkbox for rare cases or when instructed to do so by Coveo Support.

Match the query

The Match the query checkbox is cleared by default because it’s typically desirable for ART to be allowed to inject search results that don’t necessarily match the keywords entered in the search box by the end user. Therefore, we recommend selecting the Match the query checkbox only if the negative side of ART appears to be more important than the benefits or when instructed to do so by Coveo Support.

Example

Many customers on a community search page start seeking help on a popular product by entering the part number. However, many useful items available about this product don’t include that part number. Consequently, the initial search results don’t return relevant items, but customers keep adapting their keywords or navigating in search results until they find an appropriate result for the product. Because ART learns that this result was useful in relation with this part number, when the Match the query checkbox is cleared, ART can now recommend it immediately, even if the result doesn’t match the part number.

However, the Match the query checkbox must be selected for ART models associated with query pipelines that serve Coveo for Commerce interfaces. This allows the model to return only search results that match the keywords in the user’s query, increasing search relevance.

Example

During a shopping session, a customer typically browses through multiple items after performing a query.

For example, a user might search for a red t-shirt, then ultimately click a search result that depicts a blue t-shirt.

Since ART learns from searches and clicks to boost search results, the model may learn a relation between the query red t-shirt and an index item that represents a blue t-shirt.

By selecting the Match the query checkbox, you ensure search results boosted by the model match the keywords in the red t-shirt query.

Comply with Intelligent Term Detection (ITD)

Select the Comply with Intelligent Term Detection (ITD) checkbox when you want large query expressions (lq) (essentially support case descriptions) to be processed by ART to refine the query before it’s sent to the index.

This option is particularly useful when the Coveo ML model is used in case deflection panels that may include long case descriptions in their queries.

When ITD is enabled, ART selects and injects the most relevant keywords from the lq into the current basic query expression (q). By default, this selection is based on the average importance of each term, and on the longest substring from your search interface user queries that’s part of a list of top user queries. The list contains the top user queries that have been submitted at least five times each in your search interface (see How Does Intelligent Term Detection (ITD) Work?).

Model association parameters

You can use the following parameters when creating or editing a Coveo ML ART model association.

id (string)

The unique identifier of the model association (automatically generated by the Coveo Search API).

Example: 62579f33-a505-4d07-b77d-545aefb2eea1

position (integer [int32])

The position of the model in the order of execution (see Reorder model associations).

Example: 8

modelId (string)

The unique identifier of the model (see Review model information).

Example: c7ab60e2-e6b8-41e8-be6a-ad5c8edc662e

modelDisplayName (string)

The name of the model as selected when creating the model. This field is automatically filled with the name of the Coveo ML model.

Example: MyModelName

modelEngine (string)

The ID of the Coveo ML model. This field is automatically filled with the ID of the Coveo ML model.

Example: topclicks

modelStatus (string)

The status of the model. This field is automatically generated according to the current ML model status.

Example: ONLINE

condition (string)

The unique identifier of the condition that must be satisfied for a request to be processed by the ML model.

Example: c7ab60e2-e6b8-41e8-be6a-ad5c8edc662e

conditionDefinition (string)

The QPL expression that indicates the condition defined for the model association (see Query Pipeline Language (QPL)). This field is automatically filled when a condition is specified.

Example: when $searchHub is \"internalSearch\"

rankingModifier (integer [int32])

The ranking score modifier the ML model should apply to each item it recommends (see Ranking modifier).

The rankingModifier value must be a number between 0 and 1,000 (for example, 100).

Default: 250

Tip
Leading practice

When associating an ART model with a query pipeline that handles product listing queries, make sure to set the rankingModifier value to 500.

maxRecommendations (integer [int32])

The maximum number of ART-boosted items that the ML model should return in a results page.

Default: 5

Note

The maxRecommendations parameter value indicates the maximum number of ART-boosted items that the model returns before the Coveo Search API and the index apply subsequent filters (for example, query pipeline rules or item permission).
Tip
Leading practice

When associating an ART model with a query pipeline that handles product listing queries, make sure to set the maxRecommendations value to 50.

cacheMaximumAge (string)

The maximum age of cached query results the ML model should accept, in the ISO-8601 format only including the seconds and milliseconds part.

For each incoming query to be processed by the ML model, if a result set for an identical previously made query is available in the cache and this result set isn’t older than the specified value, the ML model makes recommendations based on that cached query result set. Otherwise, the query is executed against the index.

Default: PT105

intelligentTermDetection (boolean)

Whether the model should use the ITD feature to refine queries by extracting relevant keywords from the lq (Comply with Intelligent Term Detection (ITD)).

Default: false

intelligentTermDetectionPartialMatchThreshold (string)

An ART model that complies with ITD extracts refined keywords from the user’s lq using the partial match feature (see How does Intelligent Term Detection (ITD) work?).

By default, the partial match settings used for ITD has a partialMatchThreshold value of 60%.

The intelligentTermDetectionPartialMatchThreshold model association parameter allows you to modify that default value.

Default: 60%

Example: 75%

intelligentTermDetectionPartialMatchKeywords (integer [int32])

An ART model that complies with ITD extracts refined keywords from the user’s lq using the partial match feature (see How does Intelligent Term Detection (ITD) Work?).

By default, the partial match settings used for ITD has a partialMatchKeywords value of 1.

The intelligentTermDetectionPartialMatchKeywords model association parameter lets you to modify that default value, allowing you to increase the number of keywords required for the partial match feature to be activated. This means that ITD can use a larger subset of the user’s query.

Default: 1

Example: 4

matchBasicExpression (boolean)

Whether all items recommended by the ML model should match the q (see Match the query).

Default: false

matchAdvancedExpression (boolean)

Whether all items recommended by the ML model should match the advanced query expression (aq) (see Match the advanced query).

Default: true

customQueryParameters (JValue (object))

A JSON object representing the additional parameters to send to Coveo ML on all queries.

locale (string)

The locale of the current user. Adding a locale parameter to a model association allows Coveo ML to provide more relevant recommendations by taking into account the user’s language and regional preferences.

Example

The locale for a user in the United States would be: en-US.

Code sample

The following code sample shows an ART model association in JSON:

{
  "position": 1,
  "modelId": "XXXXXX_topclicks_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX",
  "condition": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
  "rankingModifier": 300,
  "intelligentTermDetection": true,
  "matchBasicExpression": true,
  "matchAdvancedExpression": true,
  "useAdvancedConfiguration": false
}

For complete information on ART model available association parameters, see ART Model association parameters reference.

Required privileges

By default, members with the required privileges can view and edit elements of the Models (platform-ca | platform-eu | platform-au) page.

The following table indicates the privileges required to use elements of the Models page and associated panels (see Manage privileges and Privilege reference).

Action Service - Domain Required access level

View model associations

Machine Learning - Models
Organization - Organization
Search - Query pipelines

View

Edit model associations

Organization - Organization
Machine Learning - Models

View

Search - Query pipelines

Edit