Reviewing Coveo Machine Learning Model Information

The Coveo Cloud Administration Console <MODEL_NAME> subpage allows members with the required privileges to understand the learning process of a specific Coveo Machine Learning (Coveo ML) model. You can use this panel to review model information, such as the number of items known per search hub and samples of top user queries.

To access this subpage, access the Models page of the Administration Console, click the desired model, and then in the Action bar, click Open.

Filter Information by Language

You can review language specific information and statistics such as the number of candidates and candidate examples to ensure that the model is behaving as expected (see Reference).

In the <MODEL_NAME> panel, under Language, click the drop-down menu, and then select the desired language.

Reference

In the <MODEL_NAME> panel, you can review model candidates, fields, associated pipelines, and statistics, depending on the model type.

Automatic Relevance Tuning (ART)

ART “General” Section

Information Definition
Model ID The unique identifier of the model.
Model version The engine version of the model followed by the model last update date in UNIX timestamp format.
Engine version The version number of the learning algorithm that was used to build the model.
Content ID keys The field used by the model to identify index items (e.g., urihash, permanentid, etc.)
Removed context keys The contexts for which the model doesn’t recommend items. Those contexts are either statistically irrelevant or their values contain too many long strings of text.1

Note 1: To see why the model excludes specific context keys, use your browser developer tools:

  1. Access your browser developer tools, and then select the Network tab.

  2. Refresh the [Model_Name] subpage, and then select the latest details HTTP request.

  3. In the Preview tab, expand the info property, and then expand the featureSelectLog property.

  4. You can now expand all context keys for which the model doesn’t recommend items. The reason for this exclusion appears next to statusMessage.

ART “Associated Pipelines” Section

The section lists the query pipelines associated with the model.

Next to each pipeline card, you can click Menu, and then select one of the two following options:

  • Edit association

    OR

  • Dissociate

Depending on your selection:

  • If you selected Edit association, on the Edit a Model Association subpage, make the desired changes, and then click Save (see ART Advanced Configuration Options).

  • If you selected Dissociate, in the Dissociate From Pipeline dialog, click Dissociate model.

ART “Model Building Statistics” Section

Statistic Definition
Click event count The total number of click events used in the model creation.
Custom event count The total number of custom events used in the model creation.
Search event count The total number of search events used in the model creation.
Segmented visits count The total number of sub-visits segmented from long visits used in the model creation.
Total queries The number of unique queries for which the model can recommend items.

ART “Language” Section

When reviewing this section, use the drop-down menu to filter information by the languages for which the model can make recommendations.

Information Definition
Candidate examples The sample of the top queries (maximum 10) for which the model could recommend items.
Context keys to items

The number of items that can be recommended within each context. In other words, the number of items that the model has seen for each context.

Coveo ML only keeps the most frequent context key/value pairs (maximum 2,000).

Filters The number of items that can be recommended, for each filter (e.g., country, region, hub, interface, tab) known by the model, for a query.
Queries The number of unique queries for which the model can recommend items per language.
Stop words The number of words removed from user queries before recommending items.
Words The number of known words after stemming and tokenization.

Event Recommendations (ER)

ER “General” Section

Information Definition
Model ID The unique identifier of the model.
Model version The engine version of the model followed by the model last update date in UNIX timestamp format.
Engine version The version number of the learning algorithm that was used to build the model.
Content ID keys The field used by the model to identify index items (e.g., urihash, permanentid, etc.).
Event group examples

The fields used to determine the similarity between users.

If the different events are grouped by visitId, the recommendations answer the question "Which pages were visited by users during similar visits?".

Recommendations The number of unique events that the model can recommend.
Recommendations per language

The number of items that can be recommended per language.

Context keys and candidates The sample of the top queries for which the model could recommend items for each listed context key.
Removed context keys

The contexts for which the model doesn't recommend items. Those contexts are either statistically irrelevant or their values contain too many long strings of text.1

Note 1: To see why the model excludes specific context keys, use your browser developer tools:

  1. Access your browser developer tools, and then select the Network tab.

  2. Refresh the [Model_Name] subpage, and then select the latest details HTTP request.

  3. In the Preview tab, expand the info property, and then expand the featureSelectLog property.

  4. You can now expand all context keys for which the model doesn’t recommend items. The reason for this exclusion appears next to statusMessage.

ER “Associated Pipelines” Section

The section lists the query pipelines associated with the model.

Next to each pipeline card, you can click Menu, and then select one of the two following options:

  • Edit association

    OR

  • Dissociate

Depending on your selection:

  • If you selected Edit association, on the Edit a Model Association subpage, make changes to the applied Condition, and then click Save.

  • If you selected Dissociate, in the Dissociate From Pipeline dialog, click Dissociate model.

ER “Model Building Statistics” Section

Statistic Definition
Click event count The total number of click events used in the model creation.
Custom event count The total number of custom events used in the model creation.
Search event count The total number of search events used in the model creation.
View event count The total number of view events used in the model creation.

Query Suggestions (QS)

QS “General” Section

Information Definition
Model ID The unique identifier of the model.
Model version The engine version of the model followed by the model last update date in UNIX timestamp format.
Engine version The version number of the learning algorithm that was used to build the model.
Candidates

The number of unique queries that the model can suggest.

When there are no candidates, the model is empty and can't return suggestions. This happens when there's not enough data to build a model (see the reference table in Reviewing Coveo Machine Learning Query Suggestion Candidates).

Candidates per filters The number of queries that can be suggested for each filter (e.g., country, region, hub, interface, tab) known by the model.
User context field

The context keys that the model can use to provide personalized query suggestions.

User cluster map The number of unique users for which the model can provide personalized query suggestions.
  • Coveo ML regroups users in 200 groups and provides personalized suggestions for each of these groups.

  • The queries and the titles of clicked items define the interests of each user. Coveo ML identifies the top 200 topics and assigns users to their main topic of interest.

Removed context keys

The contexts for which the model doesn't recommend items. Those contexts are either statistically irrelevant or their values contain too many long strings of text.1

Note 1: To see why the model excludes specific context keys, use your browser developer tools:

  1. Access your browser developer tools, and then select the Network tab.

  2. Refresh the [Model_Name] subpage, and then select the latest details HTTP request.

  3. In the Preview tab, expand the info property, and then expand the featureSelectLog property.

  4. You can now expand all context keys for which the model doesn’t recommend items. The reason for this exclusion appears next to statusMessage.

QS “Associated Pipelines” Section

The section lists the query pipelines associated with the model.

Next to each pipeline card, you can click Menu, and then select one of the two following options:

  • Edit association

    OR

  • Dissociate

Depending on your selection:

  • If you selected Edit association, on the Edit a Model Association subpage, make changes to the applied Condition, and then click Save.
  • If you selected Dissociate, in the Dissociate From Pipeline dialog, click Dissociate model.

QS “Model Building Statistics” Section

Statistic Definition
Click event count The total number of click events used in the model creation.
Custom event count The total number of custom events used in the model creation.
Filtered search event count The number of valid search events used in the model creation that have the minimum click count to be considered.
Search event count The total number of search events used in the model creation.
Candidates The number of unique queries that the model can suggest.

QS “Language” Section

When reviewing this section, use the drop-down menu to filter information by the languages for which the model can make recommendations.

Statistic Definition
Candidates The number of unique queries that the model can suggest.
Candidate examples The sample of the top queries (maximum 10) that the model could suggest.
Minimum click count The minimal number of clicks on a query suggestion that’s required for a candidate to remain in the model. The minimum is determined automatically depending on the language and the query count (see the reference table in Reviewing Coveo Machine Learning Query Suggestion Datasets).

Dynamic Navigation Experience (DNE)

DNE “General” Section

Information Definition
Model ID The unique identifier of the model.
Model version The engine version of the model followed by the date of its last update, in UNIX timestamp format.
Engine version The version number of the learning algorithm that was used to build the model.

DNE “Associated Pipelines” Section

The section lists the query pipelines associated with the model.

Next to each pipeline card, you can click Menu, and then select one of the two following options:

  • Edit association

    OR

  • Dissociate

Depending on your selection:

  • If you selected Edit association, on the Edit a Model Association subpage, make the desired changes, and then click Save (see DNE Advanced Configuration Options).
  • If you selected Dissociate, in the Dissociate From Pipeline dialog, click Dissociate model.

DNE “Model Building Statistics” Section

Statistic Definition
Click event count The total number of click events used in the model creation.
Facet select event count The total number of facet selection events used in the model creation.
Search event count The total number of search events used in the model creation.
Total queries The number of unique queries for which the model can recommend items.
Visit count The total number of visits used in the model creation.

DNE “Language” Section

When reviewing this section, use the drop-down menu to filter information by the languages for which the model can make recommendations.

Information Definition
Queries The number of unique queries per language for which the model can recommend items per language.
Top facets The sample of the top facets per language for which the model can automatically select and reorder values.
Filters The number of items that can be recommended, for each filters (e.g., country, region, hub, interface, tab) known by the model per language.

Product Recommendations (PR)

PR “General” Section

Information Definition
Model ID The model unique identifier used to troubleshoot issues (if any).
Model version The engine version of the model followed by the model last update date in UNIX timestamp format.
Engine version The version number of the learning algorithm that was used to build the model.
Content ID keys The field used by the model to identify index items (e.g., urihash, permanentid, etc.)

PR “Associated Pipelines” Section

The section lists the query pipelines associated with the model.

Next to each pipeline card, you can click Menu, and then select one of the two following options:

  • Edit association

    OR

  • Dissociate

Depending on your selection:

  • If you selected Edit association, on the Edit a Model Association subpage, make changes to the applied Condition, and then click Save (see PR Strategies Options).

  • If you selected Dissociate, in the Dissociate From Pipeline dialog, click Dissociate model.

PR “Model Building Statistics” Section

This section shows the total number of event types used to build the model. The higher the numbers are for each event type, the better.

“General events” Section
Statistic Definition
Click event count The total number of click events used in the model creation.
Custom event count The total number of custom events used in the model creation.
Search event count The total number of search events used in the model creation.
View event count The total number of view events used in the model creation.
“Commerce events” Section
Statistic Definition
Product details view count The total number of detailView events used in the model creation
Product purchased count The total number of addPurchase events used in the model creation.
Product quick view count The total number of productQuickView events used in the model creation.

PR “Submodel statistics” Section

This section shows key statistics for all strategies used by the model.

To see key statistics for a specific strategy, under Submodel statistics, in the drop-down menu, select the desired strategy.

Cart Recommender

If you selected the Cart recommender strategy, the following information is available:

Statistic Definition
Candidates examples Sample lists of product SKUs contained in the top shopping carts that the model can recommend.
Number of items The number of items that the model can recommend.
Other PR Strategies

If you selected the Interest-based, Frequently viewed together, Frequently bought together, Popular items (viewed), or Popular items (bought) strategy, the following information is available:

Statistic Definition
Candidates examples A sample of the top product SKUs known by the model.
Number of items The number of items that the model can recommend.

Required Privileges

By default, members of the Administrators and Relevance Managers built-in groups can view and edit elements of the Models page.

The following table indicates the privileges required to use elements of the Models page and associated panels (see Privilege Management and Privilege Reference).

Action Service - Domain Required access level
View models

Machine Learning - Models

Search - Query pipelines

View
Edit models

Machine Learning - Models

Edit

Search - Query pipelines

View
Recommended Articles