Adding and Managing Coveo Machine Learning Models

Coveo Machine Learning (Coveo ML) models are algorithms which leverage usage analytics data to provide contextually relevant recommendations (see Coveo Machine Learning Models).

The Models page of the Coveo Cloud administration console allows administrators and relevance managers to review and manage the Coveo Machine Learning (Coveo ML) models of a Coveo Cloud organization.

If your organization was created before April 23, 2019 and did not go through the Coveo ML migration process, models shown in the Models page of the administration console are displayed by unique configuration. Two models sharing the exact same configuration (but named differently) will only appear as one model in the Models page. The model displayed in the Models page depends on where the models sharing the same exact configuration are located:

  • When the models are in the same query pipeline, the first one that was created is displayed.
  • When the models are in separate pipelines, the model that was created in the most recent pipeline is displayed.

Access the “Models” Page

  1. If not already done, log in to the Coveo Cloud platform as a member of a group with the required privileges to manage machine learning models in the target Coveo Cloud organization.

  2. In the main menu on the left, under Machine Learning, select Models.

Prerequisites for Model Creation

Depending on the model type, ensure you meet the following requirements:

Automatic Relevance Tuning Models

  • Ensure that your Coveo Cloud organization has collected enough usage analytics data before creating an ART model (see Evaluating if Your Search Interface Produces Enough Data for ART).

    If there are less than 100 events available for your ART model, the model is empty and does not provide recommendations. With 100 events or more, your model starts learning and improving. However, it is possible to change this threshold if needed (see Editing the Number of Events Required to Build an ART Model).

    OR

  • When your search interface has more than 55 visits per day in which a manual query is followed by a click for a specific language, wait 24 hours to collect enough data, and then create a model with specific settings (detailed later in this article).

Query Suggestions Models

Ensure that your Coveo Cloud organization collects usage analytics data for the search hub on which you want to activate Query Suggestions (see Coveo Cloud Usage Analytics).

Query Suggestions predictive models are based on Coveo Cloud usage analytics data. If no usage analytics data is available, there will be no recommendations. If you recently started collecting usage analytics data, recommendations will improve as more data becomes available each time the model is retrained (see Training and Retraining).

When your search interface has low traffic, you may wonder if there will be enough usage analytics data to return relevant query suggestions. You can review which queries logged by Coveo Usage Analytics (Coveo UA) match minimal Query Suggestions requirements (see Reviewing Coveo Machine Learning Query Suggestion Candidates).

When relevant queries are missing, consider adding those queries by manually (or automatically with a script) generating usage analytics events (the queries must be sent with three different visit IDs):

  1. In the browser of your choice, in private mode, access your Coveo Cloud search interface.

  2. Without being authenticated, perform a query that you want to be suggested.

  3. Click a search result title.

  4. Close your browser.

  5. Repeat this procedure three times for each query.

  6. It may take up to 15 minutes before suggestions become available to a Coveo ML model retraining.

Event Recommendations Models

Ensure that view events and search action events are being pushed to Coveo UA for your organization (see Coveo Machine Learning Event Recommendations Deployment Overview).

When there are no view events or enough data to recommend an event, the Content Browser on which the dedicated pipeline is applied does not return any results, and the recommendation component will be empty on an hosted search page.

In a sandbox environment, chances are you will not actually get recommendations. You need to have visits on your page to recommend content relevant to the user visit history. The required data is collected each time a user visits a page containing the Page View Analytics component (see Pushing Coveo Usage Analytics View Events).

If no usage analytics data is available, there will be no recommendations. When you just start gathering data, recommendations will progressively improve as more data becomes available each time the model is retrained (see Training and Retraining).

Dynamic Navigation Experience Models

The Coveo ML Dynamic Navigation Experience (DNE) feature is only available for Coveo Cloud organizations created after May 21, 2019.

To effectively rank facets and facet values, a Coveo DNE ML model mainly learns from search and click events where end users have interacted with facets and obtained the desired result items. Therefore, the more the model learns from facet-related actions performed by your end users, the more effectively it can rank facets and provide relevant search results.

Add, Edit, and Delete Models

Add a Model

This section applies only for Coveo Cloud organizations created after April 23, 2019 and older ones that went through the Coveo ML migration process. You can contact either your Coveo Customer Success Manager (CSM) or Coveo Support if you want to proceed with the migration.

In older non-migrated organizations, you create models from the Machine Learning tab of a query pipeline configuration (see Adding and Editing Coveo Machine Learning Automatic Relevance Tuning Models in a Query Pipeline, Adding and Editing Coveo Machine Learning Query Suggestions Models in a Query Pipeline, and Adding and Editing Coveo Machine Learning Event Recommendations Models in a Query Pipeline).

  • Members with the View privilege on Models can review Coveo ML model configurations in read-only mode when clicking Open (see Reviewing Coveo Machine Learning Model Information).

  • The Coveo ML Dynamic Navigation Experience (DNE) feature is only available for Coveo Cloud organizations created after May 21, 2019.

  1. Ensure you meet the requirements before creating your model (see Prerequisites for Model Creation).

  2. On the Models page, to add a model, depending on your role:

    • (For administrators and relevance analysts) On the right-hand side of the page, click the Add Model drop-down menu, and then select Add model.

    OR

    • (For developers)
    1. On the right-hand side of the page, click the Add Model drop-down menu, and then select Add model with JSON.

    2. In the Add a Model With JSON panel that appears, enter a model configuration (see Add a Model With JSON).

    3. Click Add Model.

  3. (If you selected Add model only) In the Add a Machine Learning Model panel:

    1. In the first input, enter a meaningful display Name for the model.

      A name related to your use case will ease the management of your Coveo ML deployment.

      The name will appear in the Name column on the Models page.

      CommunitySearch

    2. Under Model type, depending on your use case, select one of the available model types: Automatic Relevance Tuning, Query Suggestions, Event Recommendations, or Dynamic Navigation Experience.

    You need to create more than one recommendation model only when you want to be able to include recommendation windows for different content types (see the note in Coveo Machine Learning Event Recommendations Deployment Overview).

    1. Click Next.

    2. In the Learning interval section, optionally change the default values:

      1. Under Frequency, select the rate at which the model is refreshed between the following values: Daily, Weekly, or Monthly. The default value is Weekly (see Training and Retraining).

        The more visits you have on your page, the higher the frequency should be.

      2. Under Data period, select the usage analytics data time interval on which the model will be based between the following values: 1 week, 1 month, 3 months, 6 months, 1 year. The default value is 3 months, meaning the last 90 days from today (see Training and Retraining).

        The more hubs, interfaces, and languages you have, the more context your model needs to learn from to provide relevant suggestions.

        Consider selecting a longer period when your search interface serves a small number of queries and want to improve the recommendations relevancy by using a larger data set.

        Consider selecting a shorter period when your search hub serves a large number of queries and you want the recommendations to be more responsive to user behavior trends.

      3. Click Next.

    3. In the Learn from section, optionally refine the data that model uses to make its recommendations depending on your use case:

      By narrowing down the set of data a model uses, you can better customize relevancy for specific user groups and use cases. These filters can be applied on all events, or on a specific event type between the following: search, click, view, or custom events.

      • You want your Query Suggestions model to return queries that pertains to a specific user group, so you add a data filter to ensure that only a specific set of analytics are used by the model for training purposes.

      • Your Community search and your Agent search have very specific vocabulary and you do not want both to influence each other in the ART model learning process, so you add a filter on the Origin 1 (Page/Hub) dimension.

      • When you want to train the model with usage analytics data from a specific search hub, consider adding a hub filter:

        1. Click the add filter icon (Icon-FilterAdditionc).

        2. Click the Select a dimension drop-down menu, and then select Origin 1 (Page/Hub) (Search, Click, Custom, and View).

        3. Click the Select an operator drop-down menu, and then select Is.

        4. Click the Select value(s) drop-down menu, and then select or type the name for your search hub. When the name of your hub is not suggested, you can type it, and then click Add “[SearchHubName]” or the Add icon (+).

        5. Click Add.

      • When you want to train the model with usage analytics data from a specific geographic origin, consider creating event filters using the Country or Region dimensions.

    4. (For Automatic Relevance Tuning, Query Suggestions, and Dynamic Navigation Experience only) Click Add Model, and then go to step 4.

      The instructions to create the model are sent to the Coveo ML service. Typically, the longer the data period, the longer it takes to build the model.

    5. (For Event Recommendations models only) Click Next.

    6. In the Recommended item types section, you can choose the type of items that the model will recommend. By default, All types is selected, so that all types of content are returned. Alternatively, you can specify some item types to filter out:

      1. Select the Specific item types radio button.

      2. Click the Add filter icon.

      3. Click the Select value(s) drop-down menu, and then select or type one or more specific content types.

        The view events pushed to Coveo UA for your organization include content of various types such as Article, Application, Download, and Course. You want to use this model to recommend only relevant downloads. You select Download.

        • The view events that are pushed to Coveo UA for your organization must contain appropriate contentType values (see Coveo Machine Learning Recommendation Content Types).

        • The View Content Type option does not behave like the other filters. All view event content types and search events are analyzed to create the model, but only pages of the specified content type(s) are recommended.

      4. Click Add.

    7. Click Add Model.

      The instructions to create the model are sent to the Coveo ML service. Typically, the longer the data period, the longer it takes to build the model.

  4. Back on the Models page, on the model row, in the Status column, the value is most probably Inactive.

    The value will change to Active when the model creation or modification is completed, typically within 30 minutes, depending on the amount of usage analytics data to process. The model can return recommendations only when its status is Active.

  5. Once the model is Active:

    1. (For ART and QS models only) On the Model Testing page, ensure it behaves as expected (see Testing Coveo Machine Learning Models).

    2. When satisfied with the results, to use it in your search experience, you need to associate it with a pipeline (see Associate a Model With a Query Pipeline).

Edit a Model

This section applies only for Coveo Cloud organizations created after April 23, 2019 and older ones that went through the Coveo ML migration process. You can contact either your Coveo Customer Success Manager (CSM) or Coveo Support if you want to proceed with the migration.

In older non-migrated organizations, you edit models from the Machine Learning tab of a query pipeline configuration (see Adding and Editing Coveo Machine Learning Automatic Relevance Tuning Models in a Query Pipeline, Adding and Editing Coveo Machine Learning Query Suggestions Models in a Query Pipeline, and Adding and Editing Coveo Machine Learning Event Recommendations Models in a Query Pipeline).

Once a model is created, you can always come back and edit its configuration.

  1. On the Models page, to edit a model, depending on your role:

    • (For administrators and relevance analysts) Click the desired model, and then in the Action bar, click Edit.

    OR

    • (For developers)

      1. Click the desired model.

      2. In the Action bar, click More, and then select Edit JSON.

      3. In the Edit a Model JSON Configuration: [Model_Name] panel that appears, make the desired changes, and then click Save (see Edit a Model JSON Configuration).

  2. (If you selected Edit only) In the Models > [Model_Name] page, in the Configuration tab, you can modify all parameter values except the Model type. Once you are done, click Save.

Your changes will be effective once the model is updated.

Delete a Model

You should dissociate the model with all query pipelines it is associated with before deleting it. A model is not automatically dissociated with pipelines once deleted.

This section applies only for Coveo Cloud organizations created after April 23, 2019 and older ones that went through the Coveo ML migration process. You can contact either your Coveo Customer Success Manager (CSM) or Coveo Support if you want to proceed with the migration.

In older non-migrated organizations, you delete models from the Machine Learning tab of a query pipeline configuration page (see Adding and Editing Coveo Machine Learning Automatic Relevance Tuning Models in a Query Pipeline, Adding and Editing Coveo Machine Learning Query Suggestions Models in a Query Pipeline, and Adding and Editing Coveo Machine Learning Event Recommendations Models in a Query Pipeline).

  1. On the Models page, click the machine learning model you want to remove.

  2. In the Action bar, click Delete.

  3. In the confirmation prompt that appears, click Confirm.

    If the model was associated to a query pipeline, the model will not be automatically dissociated (see Dissociate a Model).

Review Active Model Information

On the Models page, click the desired model (must be Active), and then in the Action bar, click Open (see Reviewing Coveo Machine Learning Model Information).

Review Model Details

On the Models page, on each row, you can review the following model information:

  • Status

    Active, Inactive, or Deleted

  • Type and current output:

    • Automatic Relevance Tuning and the number of unique Queries for which the model can recommend items

    • Query Suggestions and the number of unique queries (Candidates) that the model can suggest

    • Event Recommendations and the number of unique events (Recommendations) that the model can recommend

    • Dynamic Navigation Experience and the number of unique Queries used to build the model

      The Coveo ML Dynamic Navigation Experience (DNE) feature is only available for Coveo Cloud organizations created after May 21, 2019.

  • Refresh schedule

    The time or date of the last update and next planned update.

“Models” Page Navigation

Filter Models by Type

On the Models page, in the right section of the Action bar, click the Model type drop-down menu, and then select one of the following values: All, Automatic Relevance Tuning, Query Suggestions, Event Recommendations, or Dynamic Navigation Experience.

Search Models

On the Models page, in the right section of the Action bar, type keywords in the Filter box. You can filter models by name, type, or status.

On the Models page, at the bottom-right of the table, click the left and right arrow icons, or a page number to navigate through pages.

Set the Number of Models per Page

On the Models page, at the bottom-left of the page, select 10 or 100.

By default, the table shows 10 models per page.

Required Privileges

The following table indicates the privileges required to view and edit elements of the Models page and associated panels (see Privilege Management and Privilege Reference).

Depending on your Coveo Cloud organization creation date (see Review Organization Information):