Adding and Managing Coveo Machine Learning Models

Coveo Machine Learning (Coveo ML) models are algorithms which leverage usage analytics data to provide contextually relevant recommendations (see Coveo Machine Learning Models).

The Models page of the Coveo Cloud Administration Console allows users with the required privileges to manage and review the ML models of a Coveo Cloud organization.

Prerequisites for Model Creation

The prerequisites differ for every Coveo ML feature. Click one of the following feature links to access the corresponding prerequisites:

Prerequisite for ART Model Creation

Ensure that your Coveo Cloud organization has collected enough usage analytics data before creating an ART model.

When your search interface has more than 55 visits per day in which a user query is followed by a click for a specific language, wait until the minimum data requirement is met, and then create a model with the proper training set.

Prerequisite for QS Model Creation

Ensure that your Coveo Cloud organization collects usage analytics data for the search hub on which you want to activate QS (see Coveo Usage Analytics).

QS models are based on Coveo Cloud Platform usage analytics data, so if no data is available, there will be no suggestions. If you recently started collecting usage analytics data, suggestions will improve as more data becomes available each time the model is retrained.

When your search interface has low traffic, you may wonder if there will be enough usage analytics data to return relevant query suggestions. You can review which queries logged by Coveo Usage Analytics (Coveo UA) match minimal QS requirements (see Reviewing Coveo Machine Learning Query Suggestion Candidates).

Prerequisites for ER Model Creation

Ensure that view events and search action events are being pushed to Coveo UA for your organization.

When there are no view events or not enough data for the model to recommend content, the ER model won’t return results, and the Recommendation Component will be empty.

In a sandbox environment, you’ll probably not get any actual recommendations. You need to have visits on your page to recommend content that’s relevant to the visit history of your users. The required data is collected each time a user visits a page containing the Page View Analytics component (see Pushing Coveo Usage Analytics View Events).

If no usage analytics data is available, there will be no recommendations. If you recently started collecting usage analytics data, recommendations will improve as more data becomes available each time the model is retrained.

Prerequisites for DNE Model Creation

To effectively rank facets and facet values, a Coveo DNE ML model mainly learns from search and click events in which end users interacted with facets and obtained the desired result items. Therefore, the more the model learns from facet-related actions performed by your end users, the more effectively it can rank facets and provide relevant search results.

Prerequisite for PR Model Creation

UPCOMING

Coveo ML PR models use UA events to relevantly target and suggest products to your customers. You must thus ensure that your organization properly collects usage analytics data from your commerce interfaces.

If there are less than 100 events available for your PR model, the model is empty and doesn’t provide recommendations. With 100 events or more, your model starts to learn and improve.

Create a Model

The process of creating a Coveo ML model differs depending on the Coveo ML feature you want to add to your organization. Click one of the following feature links to access the corresponding procedure:

Advanced users may want to create a Coveo ML model using JSON.

Create an ART Model

  1. On the Models page, click the Add Model drop-down menu, and then select Add model.

  2. Enter a meaningful display Name for the model.

  3. Under Model type, select Automatic Relevance Tuning, and then click Next.

  4. (Optional) In the Learning interval section, change the default Frequency and Data period values, and then click Next.

  5. (Optional) In the Learn from section, add filters to refine the data that the model uses to make its recommendations.

  6. Click Add Model.

  7. On the Models page, under the Status column, in the model row, the value is most probably Inactive.

    This value will change to Active when the model creation or modification is complete (typically within 30 minutes, depending on the amount of usage analytics data to process). The model can only return recommendations when its status is Active.

    For more information on Coveo ML model statuses, see the “Status” column reference.

  8. Once active, you can test your model to ensure that it behaves as expected.

  9. Once you’re satisfied with the test results, you must associate the model with a pipeline to take advantage of the model in a search interface.

Create a QS Model

  1. On the Models page, click the Add Model drop-down menu, and then select Add model.

  2. Enter a meaningful display Name for the model.

  3. Under Model type, select Query Suggestions, and then click Next.

  4. (Optional) In the Learning interval section, change the default Frequency and Data period values, and then click Next.

  5. (Optional) In the Learn from section, add filters to refine the data that the model uses to make its recommendations.

  6. Click Add Model.

  7. On the Models page, under the Status column, in the model row, the value is most probably Inactive.

    This value will change to Active when the model creation or modification is complete (typically within 30 minutes, depending on the amount of usage analytics data to process). The model can only return recommendations when its status is Active.

    For more information on Coveo ML model statuses, see the “Status” column reference.

  8. Once active, you can test your model to ensure that it behaves as expected.

  9. Once you’re satisfied with the test results, you must associate the model with a pipeline to take advantage of the model in a search interface.

Create an ER Model

  1. On the Models page, click the Add Model drop-down menu, and then select Add model.

  2. Enter a meaningful display Name for the model.

  3. Under Model type, select Event Recommendations, and then click Next.

  4. (Optional) In the Learning interval section, change the default Frequency and Data period values, and then click Next.

  5. (Optional) In the Learn from section, add filters to refine the data that the model uses to make its recommendations.

  6. Click Next.

  7. (Optional) In the Recommended item types section, choose the type of items that the model will recommend.

  8. Click Add Model.

  9. On the Models page, under the Status column, in the model row, the value is most probably Inactive.

    This value will change to Active when the model creation or modification is complete (typically within 30 minutes, depending on the amount of usage analytics data to process). The model can only return recommendations when its status is Active.

    For more information on Coveo ML model statuses, see the “Status” column reference.

  10. Once active, you can test your model to ensure that it behaves as expected.

  11. Once you’re satisfied with the test results, you must associate the model with a pipeline to take advantage of the model in a search interface.

Create a DNE Model

  1. On the Models page, click the Add Model drop-down menu, and then select Add model.

  2. Enter a meaningful display Name for the model.

  3. Under Model type, select Dynamic Navigation Experience, and then click Next.

  4. (Optional) In the Learning interval section, change the default Frequency and Data period values, and then click Next.

  5. (Optional) In the Learn from section, add filters to refine the data that the model uses to make its recommendations.

  6. Click Add Model.

  7. On the Models page, under the Status column, in the model row, the value is most probably Inactive.

    This value will change to Active when the model creation or modification is complete (typically within 30 minutes, depending on the amount of usage analytics data to process). The model can only return recommendations when its status is Active.

    For more information on Coveo ML model statuses, see the “Status” column reference.

  8. Once active, you can test your model to ensure that it behaves as expected.

  9. Once you’re satisfied with the test results, you must associate the model with a pipeline to take advantage of the model in a search interface.

Create a PR Model

UPCOMING

Before creating your PR model, you should make sure that the commerce interfaces in which you want to implement the models properly collect commerce usage analytics events.

  1. On the Models page, click the Add Model drop-down menu, and then select Add model.

  2. Enter a meaningful display Name for the model.

  3. Under Model type, select Product Recommendations, and then click Next.

  4. (Optional) In the Learning interval section, change the default Frequency and Data period values, and then click Next.

  5. (Optional) In the Learn from section, add filters to refine the data that the model uses to make its recommendations.

  6. Click Add Model.

  7. On the Models page, under the Status column, in the model row, the value is most probably Inactive.

    This value will change to Active when the model creation or modification is complete (typically within 30 minutes, depending on the amount of usage analytics data to process). The model can only return recommendations when its status is Active.

    For more information on Coveo ML model statuses, see the “Status” column reference.

  8. Once the model is Active, you must associate the model with a pipeline to take advantage of the model in a search interface.

Create a Model With JSON

  1. Access the Models page.

  2. On the right-hand side of the page, click the Add Model drop-down menu, and then select Add model with JSON.

  3. In the Add a Model With JSON panel that appears, enter a model configuration.

  4. Click Add Model.

Edit a Model

Once a model is created, you can always edit its configuration.

  1. Access the Models page.

  2. Click the desired model, and then in the Action bar, click Edit.

  3. On the Models > [Model_Name] page, in the Configuration tab, you can modify all parameter values except the Model type.

  4. Once you’re done, click Save.

Advanced users can edit a model JSON configuration:

  1. Click the desired model.

  2. In the Action bar, click More, and then select Edit JSON.

  3. In the Edit a Model JSON Configuration panel that appears, make the desired changes, and then click Save.

Your changes will be effective once the model is updated.

Delete a Model

You must dissociate a model from all of the query pipelines with which it’s associated before deleting it. Models aren’t automatically dissociated from pipelines when they’re deleted.

  1. On the Models page, click the ML model that you want to remove.

  2. In the Action bar, click More, and then click Delete.

  3. In the Delete a Model panel that appears, click Delete model.

Review Active Model Information

On the Models page, click the desired model (must be Active), and then in the Action bar, click Open (see Reviewing Coveo Machine Learning Model Information).

Reference

In this section:

“Status” Column

On the Models page of the Administration Console, the Status column indicates the current state of your Coveo ML models.

The following table lists the possible model statuses and their definitions:

Status Definition
Active The model is active and available.
Inactive The model is not available.
Waiting The model is in the building queue.
Building The model is currently being processed.
Failed The model couldn't be built with the requested configuration.

See Edit a Model for information on how to edit a model configuration.

Update in queue Waiting to process a scheduled update or configuration change.
Updating The model is being rebuilt based on a new configuration.
Update failed The model couldn't be updated with the requested configuration.
Unknown An error prevented the model to build successfully.

“Learning Interval” Section

In this section, you can modify the following:

  • Frequency: The rate at which the model is retrained.

  • Data period: The usage analytics data time interval on which the model will be based.

  • The more visits you have on your page, the higher the Frequency should be.

  • The more hubs, interfaces, and languages you have, the longer the Data period should be.

  • Consider selecting a longer Data period when your search interface serves a small number of queries and you want to improve the relevancy of recommendations by using a larger data set.

  • Consider selecting a shorter Data period when your search hub serves many queries and you want the recommendations to be more responsive to trends in user behavior.

For more information on Coveo ML data learning behavior, see Learning and Retraining.

“Learn From” Section

The Learn From section allows you to refine the data that the model uses to make its recommendations. By narrowing down the set of data that a model uses, you can better customize relevancy for specific user groups and use cases. You can apply filters on all events, or on every event that belongs to a specific category (i.e., search, click, view, or custom events).

  • You want your QS model to return queries that pertain to a specific user group, so you add a data filter to ensure that only a specific set of analytics are used by the model for training purposes.

  • Your Community search and Agent search have very specific vocabulary. You don’t want them to influence one another in the ART model learning process, so you add a filter on the Origin 1 (Page/Hub) dimension.

To add a filter

  1. Click Add filter Filter-Addition-Icon.

  2. In the Select a dimension drop-down menu, select the dimension on which you want to base the learning of the model.

  3. Click the Select an operator drop-down menu, and then select the appropriate operator.

  4. Click the Select value(s) drop-down menu, and then add, type, or select the appropriate value.

  5. Click Add Filter.

In the Recommended Item Types section, you can choose the types of items that an ER model recommends. By default, All types is selected, meaning that all types of content are returned. Alternatively, you can choose specific item types to filter out.

To filter out specific item types

  1. Select the Specific item types radio button.

  2. Click Add filter Filter-Addition-Icon.

  3. Click the Select value(s) drop-down menu, and then select or type one or more specific content types.

    The view events that are pushed to Coveo UA for your organization include various types of content, such as Article, Application, Download, and Course. You only want to use this model to recommend relevant downloads, so you select Download.

    • The view events that are pushed to Coveo UA for your organization must contain appropriate contentType values.

    • The View Content Type option doesn’t behave like the other filters. All view event content types and search events are analyzed to create the model, but only pages of the specified content types are recommended.

    • Only create more than one recommendation model if you want to include recommendation windows for different content types (see the note in Coveo Machine Learning Event Recommendations Deployment Overview).

  4. Click Add Filter.

ART Reference

Default number of recommended results: 5

ART recommends the 5 best learned search results, but may recommend less than 5 when:

  • The query isn’t very frequent, and less than 5 items were learned for that query.

  • Some recommendations are secured and not accessible to the user performing the query.

  • Some recommendations are filtered out because they don’t match filters such as the current facet selections.

Facts

  • Following an empty query, ART returns the most clicked items during the data period of the model.

  • ART can inject items that would’nt normally be included in search results because they were learned to be relevant even if they don’t contain some or all of the searched keywords. This is one of the key ART benefits as a user can find the most useful items without having to type the right keywords or the specific synonym contained these items. This is the default behavior (the Match the Query parameter default value is false).

  • ART currently ignores all special characters or operators in the user entered query to only keep the keywords and therefore ignores the special behavior described in Using Special Characters in Queries or Search Prefixes and Operators.

  • This ART behavior may lead to unexpected search results when users want to take advantage of more advanced query syntax.

    A user is searching for items containing a specific phrase by entering the phrase enclosed in double-quotes (see Searching for a Phrase) and no items contain the phrase. The user would expect to get no search results, but ART can inject items that were clicked for similar queries made of one or some of the searched keywords (not in a phrase search).

QS Reference

Default number of suggestions: 10

The JavaScript Search Framework’s Omnibox component overrides this value at query time with its own default of 5 (see numberOfSuggestions option). Thus, in a JS Search Framework-powered search interface, a maximum of 5 (not 10) query suggestions is recommended by default.

ER Reference

Default number of recommendations: 10

While the default numberOfResults is 10, you typically don’t want to display 10 recommendations. The leading practice is to limit the number of recommendations to 5. In other words, when you configure a Recommendation component in a JS Search Framework-powered search interface, explicitly set the resultsPerPage option to 5 (see Coveo Machine Learning Event Recommendations Deployment Overview).

DNE Reference

  • Default number of expanded facets: 4

  • Default number of displayed values in expanded facets: 8

While you can change both values, it’s not recommended and requires developer skills.

Required Privileges

By default, members of the Administrators and Relevance Managers built-in groups can view and edit elements of the Models page.

The following table indicates the privileges required to use elements of the Models page and associated panels (see Privilege Management and Privilege Reference).

Action Service - Domain Required access level
View models

Machine Learning - Models

Search - Query pipelines

View
Edit models

Machine Learning - Models

Edit

Search - Query pipelines

View
Recommended Articles