Associate an Automatic Relevance Tuning Model With a Query Pipeline

When a Coveo Machine Learning (Coveo ML) Automatic Relevance Tuning (ART) model has been created, it must be associated with a query pipeline to be effective in a search interface.

Organization members with the required privileges can access the Machine Learning tab of a query pipeline configuration page to manage Coveo ML model associations for that query pipeline.

Leading Practices

Duplicate the Production Query Pipeline

Once you have created a Coveo ML ART model, the leading practice is to duplicate the query pipeline with which you plan to associate the model, and then associate the model with the pipeline copy.

Once satisfied with the model efficiency on the pipeline copy, dissociate the model from the test pipeline and associate it with the production pipeline. You can test your Coveo ML model’s efficiency by performing A/B tests or model testings.

Once you’re done testing the model in the test pipeline, you can delete the test pipeline.

Validate That the Model Is Effective

To validate that your Coveo ML models work as expected, you can inspect your models.

Plan the Usage of Custom Contexts

While the various Coveo ML features can perform well without custom context information, using custom contexts can take Coveo ML relevance one step further.

You can define custom contexts and then pass appropriate ones along with usage analytics events and queries to allow Coveo ML to take them into account.

If you’re just getting started with Coveo ML, you can skip this step to take advantage of Coveo ML features more quickly and easily, and consider using custom contexts in a second phase.

Associate an ART Model

Follow the model association leading practices when associating your model with your query pipeline.

  1. On the Query Pipelines page, click the query pipeline for which you want to associate a ART model, and then in the Action bar, click Edit Components.

  2. On the subpage that opens, select the Machine Learning tab, and then in the upper-right corner, click Associate Model.

  3. In the Model drop-down menu, select the desired model.

  4. On the right side, under Condition, you can optionally select a query pipeline condition in the drop-down menu or create a new one.

  5. In the Advanced Configuration section, you can optionally select or deselect the following parameters:

  6. Click Associate Model.

Edit an ART Model Association

Follow the model association leading practices when associating your model with your query pipeline.

  1. On the Query Pipelines page, click the query pipeline for which you want to edit a model association, and then in the Action bar, click Edit Components.

  2. On the subpage that opens, click the desired model, and then in the Action Bar, click Edit.

  3. On the right side, under Condition, you can optionally select a query pipeline condition in the drop-down menu or create a new one.

  4. In the Advanced Configuration section, you can optionally select or deselect the following parameters:

  5. Click Save.

Associate an ART Model via a JSON Configuration

Advanced users may want to manage a model association via a JSON configuration to specify association parameters that don’t fit with the parameters available in the Administration Console.

  1. On the Query Pipelines page, click the query pipeline for which you want to associate a model, and then in the Action bar, click Edit Components.

  2. On the subpage that opens, select the Machine Learning tab, and then in the upper-right corner, click menu-button, and select Associate a model in JSON view.

  3. On the Associate a Model subpage, in JSON view, replace the <Model_ID> placeholder with the actual ID of the model you want to associate with the pipeline (see Review Coveo Machine Learning Information).

    Once you have accessed the Associate a Model subpage in JSON view:

    • You can always go back to the Associate a Model subpage in the UI view and use the available options. However, all unsaved changes made in the Associate a Model subpage in JSON view will be lost.

    • The Associate a Model subpage in JSON view becomes the default model association view for that model. In other words, the Associate a Model subpage in JSON view is now automatically displayed when you access this model association.

  4. Click Associate Model.

While you can specify model association parameters when creating a model association via a JSON configuration, the leading practice is to first create the association, and then edit its configuration if needed. This allows you to easily tweak the entire configuration of the model association instead of creating it from scratch.

Edit an ART Model Association via a JSON Configuration

Advanced users may want to manage a model association via a JSON configuration to specify association parameters that don’t fit with the parameters available in the Administration Console.

  1. On the Query Pipelines page, click the query pipeline for which you want to edit a model association, and then in the Action bar, click Edit Components.

  2. In the Machine Learning tab, click the desired model, and then in the Action bar, click Edit.

  3. On the Edit a Model Association subpage, click menu-button, and then select Switch to JSON view.

  4. On the Switch to JSON view? panel that appears, click Switch to JSON view.

    Switching to the JSON view of the Edit a Model Association subpage cancels unsaved configuration changes made on the Model Association page.

  5. On the Edit a Model Association subpage, in JSON view, tune the JSON model association configuration as needed (see Model Association Parameters).

    Once you have accessed the Associate a Model subpage in JSON view:

    • You can always go back to the Edit a Model Association subpage in the UI view and use the available options. However, all unsaved changes made in the Associate a Model subpage in JSON view will be lost.

    • If you specified non-default parameters in JSON view of the Edit a Model Association subpage, the complete configuration will be reset to the default one when switching back to the UI view of the Edit a Model Association subpage.

    • The Edit a Model Association subpage in JSON view becomes the default model association view for that model. In other words, the Edit a Model Association subpage in JSON view is now automatically displayed when you access this model association.

  6. Click Save.

Dissociate a Model

  1. On the Query Pipelines page, click the query pipeline from which you want to dissociate a model, and then in the Action bar, click Edit Components.

  2. On the subpage that opens, select the Machine Learning tab.

  3. In the Machine Learning tab, click the model you want to dissociate from the pipeline, and then in the Action Bar, click Dissociate.

The model is now dissociated from the pipeline.

Reorder Model Associations

The Coveo ML models of a given type are executed in the order in which they appear in the page until a condition is satisfied.

The first model on the list will be used if no conditions are met.

  1. On the Query Pipelines page, click the query pipeline in which you want to reorder model associations, and then in the Action bar, click Edit Components.

  2. On the subpage that opens, select the Machine Learning tab.

  3. In the Machine Learning tab of the desired query pipeline, click the model whose position you want to change.

  4. In the Action bar, use the Move up or Move down arrows to change the position of the model.

Reference

ART Advanced Configuration Options

Ranking Modifier

The Recommended value of 250 typically modifies the ranking score of the ART results so that they appear among the top 10 search results.

You should only consider reducing the default value when ART result ranking scores significantly exceed the default ranking, such as when ART results are always appearing among the top five results. You generally don’t want ART results to completely override the normal search results ranking, but rather bring them among the first top 10 results. Typical ranking modifiers are between 50 and 250.

Oppositely, you should only consider increasing the default value when the ART results generally don’t appear within the first search result page. Increase the value until the ART results surface among the top 10 results.

ART results can be buried lower in search results when other query pipeline rules such as Featured Results and Ranking Expressions, or hard-coded query ranking expressions (QREs) significantly modify result ranking scores. When this is the case, re-evaluate and consider eliminating such query pipeline rules or hard-coded expressions.

Match the Advanced Query

The Match the advanced query check box is selected by default because it’s typically desirable for ART results to match search interface scope, facet selections, and filters. Items recommended by ART are typically not distinguishable from other results in search interfaces. If ART is allowed to inject results outside of the expected scope, the overall search experience may feel confusing for your users.

EXAMPLE

On a community search page, an end user selects the Knowledge Base Article value in the Item Type facet, enters acme television set in the search box, and then submits the query.

In the query pipeline, the query is processed by an ART model whose Match the advanced query option is enabled. Consequently, the model can only recommend knowledge base articles (i.e., items whose @documenttype field value is equal to kbarticle).

If the option was disabled, the model could also recommend items of a different type, such as videos and user manuals. However, the end user likely expects such items to be filtered out, resulting in a confusing experience.

You should only consider clearing the Match the advanced query check box for rare cases or when instructed to do so by Coveo Support.

Match the Query

The Match the query check box is cleared by default because it’s typically desirable for ART to be allowed to inject search results that don’t necessarily match the keywords entered in the search box by the end user.

EXAMPLE

Many customers on a community search page start seeking help on a popular product by entering the part number. However, many useful items available about this product don’t include that part number. Consequently, the initial search results don’t return relevant items, but customers keep adapting their keywords or navigating in search results until they find an appropriate result for the product.

Because ART learns that this result was useful in relation with this part number, when the Match the query check box is cleared, ART can now recommend it immediately, even if the result doesn’t match the part number.

Consider selecting the Match the query check box only if the negative side of ART appears to be more important than the benefits or when instructed to do so by Coveo Support.

Comply With Intelligent Term Detection (ITD)

Select the Comply with Intelligent Term Detection (ITD) check box when you want large query expressions (essentially support case descriptions) to be processed by ART to refine the query before it’s sent to the index.

This option is particularly useful when the Coveo ML model is used in case deflection panels that may include long case descriptions in their queries.

When ITD is enabled, ART selects and injects the most relevant keywords from the large query expression into the current basic query expression (q). By default, this selection is based on the average importance of each term, and on the longest substring from your search interface user queries that’s part of a list of top user queries. The list contains the top user queries that have been submitted at least five times each in your search interface (see How Does Intelligent Term Detection (ITD) Work?).

Model Association Parameters

You can use the following parameters when creating or editing a Coveo ML ART model association.

id (string)

The unique identifier of the model association (automatically generated by the Coveo Search API).

Example: 62579f33-a505-4d07-b77d-545aefb2eea1

position (integer [int32])

The position of the model in the order of execution (see Reorder Model Associations).

Example: 8

modelId (string)

The unique identifier of the model (see Reviewing Coveo Machine Learning Model Information).

Example: c7ab60e2-e6b8-41e8-be6a-ad5c8edc662e

modelDisplayName (string)

The name of the model as selected when creating the model. This field is automatically filled with the name of the Coveo ML model.

Example: MyModelName

modelEngine (string)

The ID of the Coveo ML model (see Creating a Model With JSON). This field is automatically filled with the ID of the Coveo ML model.

Example: topclicks

modelStatus (string)

The status of the model. This field is automatically generated according to the current ML model status.

Example: ONLINE

condition (string)

The unique identifier of the condition that must be satisfied for a request to be processed by the ML model.

Example: c7ab60e2-e6b8-41e8-be6a-ad5c8edc662e

conditionDefinition (string)

The QPL expression that indicates the condition defined for the model association (see Query Pipeline Language (QPL)). This field is automatically filled when a condition is specified.

Example: when $searchHub is \"internalSearch\"

rankingModifier (integer [int32])

The ranking score modifier the ML model should apply to each item it recommends (see Ranking Modifier).

The rankingModifier value must be a number between 0 and 1,000 (e.g., 100).

Default: 250

maxRecommendations (integer [int32])

The maximum number of ART-boosted items that the ML model should return in a results page.

Default: 5

The maxRecommendations parameter value indicates the maximum number of ART-boosted items that the model returns before the Coveo Search API and the index apply subsequent filters (e.g., query pipeline rules or item permission).

cacheMaximumAge (string)

The maximum age of cached query results the ML model should accept, in the ISO-8601 format only including the seconds and milliseconds part.

For each incoming query to be processed by the ML model, if a result set for an identical previously made query is available in the cache and this result set isn’t older than the specified value, the ML model makes recommendations based on that cached query result set. Otherwise, the query is executed against the index.

Default: PT105

intelligentTermDetection (boolean)

Whether the model should use the Intelligent Term Detection (ITD) feature to refine queries by extracting relevant keywords from the large query expression (lq) (Comply With Intelligent Term Detection (ITD)).

Default: false

matchBasicExpression (boolean)

Whether all items recommended by the ML model should match the basic query expression (q) (see Match the Query).

Default: false

matchAdvancedExpression (boolean)

Whether all items recommended by the ML model should match the advanced query expression (aq) (see Match the Advanced Query).

Default: true

customQueryParameters (JValue (object))

A JSON object representing the additional parameters to send to Coveo ML on all queries.

useAdvancedConfiguration (boolean)

Whether the model association specifies an advanced configuration.

Default: false

Code Sample

The following code sample shows an ART model association in JSON:

{
  "position": 1,
  "modelId": "XXXXXX_topclicks_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX",
  "condition": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX",
  "rankingModifier": 300,
  "intelligentTermDetection": true,
  "matchBasicExpression": true,
  "matchAdvancedExpression": true,
  "useAdvancedConfiguration": false
}

For complete information on ART model available association parameters, see ART Model Association Parameters Reference.

Required Privileges

By default, members of the Administrators and Relevance Managers built-in groups can view and edit elements of the Models page.

The following table indicates the privileges required to use elements of the Models page and associated panels (see Manage Privileges and Privilege Reference).

Action Service - Domain Required access level

View model associations

Machine Learning - Models

Search - Query pipelines

View

Edit model associations

Machine Learning - Models

Search - Query pipelines

Edit

Recommended Articles