Managing the Coveo Machine Learning Model Associations With Query Pipelines

Members of the Administrators and Relevance Managers built-in groups can access the Machine Learning tab of a query pipeline configuration page to manage Coveo Machine Learning (Coveo ML) model associations for that query pipeline (see Coveo Machine Learning Models).

This article applies only for Coveo Cloud organizations created after April 23, 2019 and older ones that went through the Coveo ML migration process. You can either contact your Coveo Customer Success Manager (CSM) or Coveo Support if you want to proceed with the migration.

Duplicate the Production Query Pipeline

Once you created a Coveo ML model and tested it on the Model Testing page, the leading practice is to duplicate the query pipeline in which you plan to associate the model, and then associate the model with the pipeline copy.

Access the “Machine Learning” Tab of a Query Pipeline

  1. On the Query Pipelines page:

    • Click the query pipeline to edit and then in the Action bar, click Edit components.

    OR

    • Double-click the query pipeline to edit.
  2. In the Query Pipeline > [Name] page, select the Machine Learning tab.

Associate a Model With a Query Pipeline

  1. Access the “Machine Learning” tab of the query pipeline that you want to associate a Coveo ML model with.

  2. In the Machine Learning tab, click Associate model.

  3. In the Associate a Machine Learning Model dialog that appears:

    Admin-QPL-ModelAssociationEx

    1. Click the first drop-down list menu, and then select the Model you want to associate.

      If you associate an Event Recommendations model with a pipeline, ensure the pipeline is not associated with any ART, QS, and DNE models.

      Associate all Event Recommendations models with the same pipeline and assess each ER model a proper condition.

      • Model display names are followed by the model type abbreviations (QS for Query Suggestions, ART for Automatic Relevance Tuning, ER for Event Recommendations, and DNE for Dynamic Navigation Experience).

      • A model can be associated with different pipelines at the same time.

    2. (Optional) Click the second drop-down list menu, and then select a Condition if you want the model to apply only when a query satisfies that condition. You can also Create a condition (see Create a Condition).

    3. When associating a QS or an ER model, go to step 4.

    4. (When associating an ART model only) In the Impact on relevancy section:

      1. Adjust the Ranking modifier slider as desired.

      2. Under Injected search results, select check boxes as needed to determine whether items recommended by the ART model should:

        • Match the advanced query

        • Match the query

          AND

        • Comply with Intelligent Term Detection (ITD)

      See ART Impact on Relevancy Options.

    5. (When associating a DNE model only) In the Advanced configuration section, optionally select check boxes to determine whether the DNE model should influence:

      • Facet ordering

      • Facet value ordering

        AND

      • Ranking boost

      If selected, you can adjust the ranking boost applied to search results that match relevant facet values.

      See DNE Advanced Configuration Options.

  4. Click Associate Model.

    The association is now effective.

  5. (For DNE models only) Configure dynamic facets in your search interface.

  6. Validate that the model is effective, for instance for ART and QS models, you can use the Content Browser page to test the query pipeline associated with the model.

  7. Tweak the model options until you are satisfied with the results, for instance for ART models, validate that ART results appear in the first 10 search results.

    If your QS model is not behaving as expected, see Getting Query Suggestions - Troubleshooting.

  8. Once satisfied, dissociate the model from the test pipeline and associate it with the production pipeline.

  9. Validate that the model works on your production site.

  10. Delete the test pipeline (see Delete a query pipeline).

  11. Optionally, plan the usage of custom contexts to improve the model relevance:

    While the various Coveo ML features can perform quite well without custom context information, using custom contexts appropriate for your business can take Coveo ML relevance one step further.

    If you are just starting using Coveo ML in your solution, you can skip this step to take advantage of Coveo ML features more quickly and easily, and consider using custom contexts in a second phase.

    You can define custom contexts and then pass appropriate ones along with usage analytics events and queries to allow Coveo ML to take them into account (see Leveraging Custom Contexts in Coveo Machine Learning Features).

Validate That ART Results Appear in the First 10 Search Results

Assuming you followed the recommendation to associate the ART model with a duplicated test pipeline:

  1. Open the search hub using the original pipeline, select a specific interface if there is more than one, and perform one of the most frequent queries as reported by your usage analytics data.

  2. Open the same search hub using the test pipeline (using the pipeline query parameter), select the interface if there is more than one, and perform the same query.

  3. Inspect the two search result sets side-by-side to look for differences in the top results ordering.

    • Results must be ordered by descending score (i.e., relevance), otherwise the ART model will not recommend results.

    • In a JavaScript search interface, you can identify an ART result as follows:

      1. Hold Alt and double-click anywhere in search result you suspect is an ART result.

        The JavaScript Search debug window opens.

      2. In the debug window, in the result section, look for the rankingModifier parameter, which is set to Reveal ART when the search result is an ART result.

        JsSearchDebugIsRecommendation1a

        OR

        Select the Highlight recommendation check box to highlight results boosted by an ART model.

  4. When ART results for a frequent query do not appear within the first search results page among the top 10 results, consider adjusting the Ranking Modifier value.

Edit a Model Association

  1. In the Machine Learning tab of the desired query pipeline:

    • Click the model association to edit, and then in the Action bar, click Edit.

      OR

    • Double-click the model association to edit.

  2. In the Edit a Machine Learning Model Association dialog, you can edit all the available parameters except the selected Model (see ART Impact on Relevancy Options or DNE Advanced Configuration Options).

  3. Click Save.

    All changes are now effective.

Dissociate a Model

  1. In the Machine Learning tab of the desired query pipeline, click the model that you want to dissociate.

  2. In the Action bar, click Dissociate.

    The model immediately ceases being associated with the query pipeline.

Reorder Model Associations

The Coveo ML models of a given type are executed in the order in which they appear in the page until a condition is satisfied.

The first model on the list will be used if no conditions are met.

To reorder model associations in a query pipeline:

  1. In the Machine Learning tab of the desired query pipeline, click a model, and then in the Action bar, click Reorder.

  2. In the Reorder a Model dialog that appears:

    1. Use the minus (-) and plus (+) buttons or enter the desired rank in the input to change the model position.

    2. Click Reorder model.

  3. Repeat step 1 and 2 until all models appear in the order of your choice.

ART Impact on Relevancy Options

Ranking Modifier

The Recommended value of 250 typically modifies the ranking score of the ART results so that they appear among the top 10 search results.

  • Consider reducing the value when ART result ranking scores significantly exceed that of the normal top results, such as when ART results are always appearing among the top five results. You generally do not want ART results to completely override the normal search results ranking, but rather bring them among the first top 10 results. Typical ranking modifiers are between 50 and 250.

  • Consider increasing the value when the ART results generally do not appear within the first search result page. Increase the value until the ART results surface among the top 10 results.

ART results can be buried lower in search results when other query pipeline rules such as Featured Results and Ranking Expressions, or hard-coded query ranking expressions (QREs) significantly modify result ranking scores. When this is the case, reevaluate and consider eliminating such query pipeline rules or hard-coded expressions.

Match the Advanced Query

The Match the advanced query check box is selected by default because it is typically desirable for ART results to match search interface scope, facet selection, and filters.

Consider clearing the Match the advanced query check box only for rare cases when instructed to do so by Coveo Support.

Items recommended by ART are typically not distinguishable from other results in search interfaces. If ART is allowed to inject results outside of the expected scope, the overall search experience may feel confusing for the end users.

On a community search page, an end user selects the Knowledge Base Article value in the Item Type facet, enters the acme television set keywords in the search box, and then submits the query.

In the query pipeline, the query is processed by an ART model whose Match the advanced query option is enabled. Consequently, the model can only recommend knowledge base articles (i.e., items whose @documenttype field value is equal to kbarticle).

If the option was disabled, the model could also recommend item types such as videos and user manuals. However, the end user likely expects such items to be filtered out, resulting in a confusing experience.

Match the Query

The Match the query check box is cleared by default because it is typically desirable for ART to be allowed to inject search results that do not necessarily match the keywords entered in the search box by the end user.

The scope broadening bonus coming with the injection of search results not matching the query may have a downside. Users may be confused (such as when someone searches for a specific keyword or an exact phrase), because the ART boosted results do not necessarily contain all of the keywords, these keywords are not highlighted in the excerpt, and are not navigable in the Quick View. The user may think that something is wrong with the search result ranking.

Consider selecting the Match the query check box only if the negative side of ART appears to be more important than the benefits or when instructed to do so by Coveo Support.

  • Even if a query contains a typographical mistake, ART still recommends results.

  • The Match the query option allows Coveo ML to associate keywords used by end users with those used in the documentation.

    Your users are searching for broken clip, but your knowledge articles refer to a clip as a latch. If the Match the query check box is cleared, Coveo ML will learn that clip and latch are the same object.

  • Depending on your use case, you could want to match the queries.

    In the People tab of your internal search interface where employees search for people names, you want all results to fully match the name they entered in the search box.

Many customers on a community search page start seeking help on a popular product by entering the part number. However, many useful items available about this product do not include that part number. Consequently, the initial search results do not return relevant items, but customers keep adapting their keywords or navigating in search results until they find an appropriate item for the product.

Because ART learns that this item was useful in relation with this part number, when the Match the query check box is cleared, ART can now recommend it immediately, even if the part number is not present in the item.

You could achieve a similar outcome using a query pipeline thesaurus rule expanding the part number with a product name that you know is present in most useful items about this product, but you would probably have to spend a significant amount of time to analyze usage analytics data to identify the issue and manually enter an appropriate thesaurus rule. ART takes care of this for you by learning from end-user behavior.

Comply with Intelligent Term Detection (ITD)

Select the Enable Intelligent Term Detection (ITD) check box when you want large query expressions (essentially support case descriptions) to be processed by ART to refine the query before it is sent to the index.

When ITD is enabled, ART selects and injects the most relevant keywords from the large query expression into the current basic query expression (q). By default, this selection is based on the average importance of each term, and on the longest substring from your search interface user queries that is part of a list of top user queries. The list contains the top user queries that have been submitted at least five times each in your search interface (see How Does Intelligent Term Detection (ITD) Work?).

This option is particularly useful when the Coveo ML model is used in case deflection interfaces that may include long case descriptions in their queries.

DNE Advanced Configuration Options

Facet Ordering

The Facet ordering check box is selected by default because it is typically desirable for DNE models to display the most relevant facets first. You can however deselect this check box if you want to take advantage of the other DNE model options, without having your facets automatically ranked.

Facet Value Ordering

The Facet Value Ordering check box is selected by default because it is typically desirable for DNE models to display the most relevant facet values within a given facet first. You can however deselect this check box if you want to take advantage of the other DNE model options, without having your facet values automatically ranked.

Ranking Boost

The Ranking boost check box is selected by default because it is typically desirable for DNE models to boost items that match facet values determined as relevant by the model. You can however:

  • Deselect this check box if you want to take advantage of the other DNE model options, without having search results that correspond to relevant facets boosted.

  • Adjust the default (recommended) boosting value applied to items matching relevant facet values.

    Increasing the ranking modifier above the recommended value can considerably alter your search results.

Required Privileges

The following table indicates the required privileges to view or edit elements of the Query Pipelines page and associated panels (see Privilege Management and Privilege Reference).

Action Service - Domain Required access level
View model associations

Organization - Organization

Search - Query pipelines

Machine Learning - Models

View
Edit model associations Organization - Organization View

Search - Query pipelines

Machine Learning - Models

Edit