Create and manage Passage Retrieval (CPR) models

This is for:

Developer System Administrator
Important

Passage Retrieval (CPR) is a paid product extension. Contact Coveo Sales or your Account Manager to add CPR to your organization license.

A Passage Retrieval (CPR) model retrieves the most relevant passages that will be used by a generative application to generate output.

To implement CPR, you must first create a CPR model.

What does a CPR model do?

When a CPR model builds, it creates embeddings for the indexed items specified in the model settings and stores the embeddings in model memory.

Note

The model is preconfigured to rebuild and update the embeddings weekly based on when the model is created. Contact your Coveo Account Manager if a different build interval is required.
Passage Retrieval passage embedding | Coveo

A CPR model uses a pre-trained sentence transformer language model to create the embeddings. The language model does this by capturing relationships between words, phrases, and sentences in the dataset.

A CPR model creates embeddings only for the content in an item's body, which is the content mapped to the body field in the Coveo index. For more information, learn how CPR uses your content.

As shown in the following diagram, the model uses chunks to create the embeddings. Instead of creating a vector for each individual word, a vector is created for a segment of text (chunk) to increase relevance.

Vector space | Coveo

As shown in the following diagram, when a user enters a query in a LLM-powered application, the CPR model embeds the query in the embedding vector space in its memory to find the most relevant passages (chunks).

In the context of the CPR passage retrieval flow, this is referred to as second-stage content retrieval. Only the passages corresponding to the most relevant items identified during first-stage content retrieval (items retrieved by the Coveo search engine) are considered. The most relevant passages are then exposed by the Passage Retrieval API to the LLM so it can generate an answer.

Vector space query | Coveo
Tip

The embeddings that are created by the CPR model aren’t impacted by usage analytics events. However, a CPR model leverages the most relevant items retrieved by your Coveo-powered search for a given user query. Therefore, by enabling an Automatic Relevance Tuning (ART) model, which learns from usage analytics events, then the most relevant items, and by extension the generated output by your LLM, will be influenced by usage analytics events.
Note

You can set a custom value for the maximum number of items that the CPR model considers when retrieving the most relevant segments of text (passages). This is an advanced model query pipeline association configuration that should be used by experienced Coveo administrators only. For more information, see CPR model association advanced configuration.

Prerequisites

Create a CPR model

  1. Depending on whether models have already been created in your Coveo organization:

  2. Click Next.

  3. In the Learn from section, select the content that the model will use. You can select the sources and apply additional filters using the Standard configuration, or use Advanced mode to define a custom filter expression.

    Important

    You’ll lose the current mode settings when you switch between Standard and Advanced mode.
    Note

    An optimal Passage Retrieval (CPR) implementation includes both a CPR model and a Semantic Encoder (SE) model. The same SE model can be used with multiple CPR models. For best results, both the CPR and SE models should be configured to use the same content. See CPR overview for information on how CPR and SE work together in the context of a user query to retrieve passages.
    Tip

    The Data volume preview section shows the impact of your settings on the data that’s available to the model.

    • In the Standard tab:

      1. In the Sources dropdown menu, select the sources that contain the items from which you want the model to learn.

        Note

        If your Coveo organization includes multiple indexes, the model can learn only from sources that are linked to the default index.

      2. (Optional) In the Apply filters on dataset section, you can specify a condition to segment the content on which the model should base its training:

        Example

        You want the model to base its training only on items for which the collection field have the FAQ value.

        Therefore, you add a collection is equal to FAQ condition.

        1. Click Add filter(s).

        2. In the Field name input, enter the name of the field that you want to use to segment the dataset.

        3. In the Select an operator dropdown menu, select the desired operator.

        4. In the Value input, enter the value of the field on which you want to segment the dataset.

        5. Click Apply.

    • In the Advanced tab:

      1. Enter a custom filter expression using Coveo query syntax.

      2. Click Apply.

  4. Click Next.

  5. In the Name your model input, enter a meaningful display name for the model, and then click Start building.

  6. You can then associate the model with a pipeline to take advantage of the model in a search interface.

Edit a CPR model

  1. On the Models (platform-ca | platform-eu | platform-au) page, click the model you want to edit, and then click Edit in the Action bar.

  2. On the subpage that opens, select the Configuration tab.

  3. In the upper-right corner, click Edit.

  4. Under Name, edit the model’s display name.

  1. Use the Project selector to associate your SE model with one or more projects.

  2. In the Learn from section, select the content that the model will use. You can select the source(s) and apply additional filters using the Standard configuration, or use Advanced mode to define a custom filter expression.

    Important

    You’ll lose the current mode settings when you switch between Standard and Advanced mode.
    Note

    An optimal Passage Retrieval (CPR) implementation includes both a CPR model and a Semantic Encoder (SE) model. The same SE model can be used with multiple CPR models. For best results, both the CPR and SE models should be configured to use the same content. See CPR overview for information on how CPR and SE work together in the context of a user query to retrieve passages.
    Tip

    The Data volume preview section shows the impact of your settings on the data that’s available to the model.

    • In the Standard tab:

      1. In the Sources dropdown menu, select the sources that contain the items from which you want the model to learn.

        Note

        If your Coveo organization includes multiple indexes, the model can learn only from sources that are linked to the default index.

      2. (Optional) In the Apply filters on dataset section, you can specify a condition to segment the content on which the model should base its training:

        Example

        You want the model to base its training only on items for which the collection field have the FAQ value.

        Therefore, you add a collection is equal to FAQ condition.

        1. Click Add filter(s).

        2. In the Field name input, enter the name of the field that you want to use to segment the dataset.

        3. In the Select an operator dropdown menu, select the desired operator.

        4. In the Value input, enter the value of the field on which you want to segment the dataset.

        5. Click Apply.

    • In the Advanced tab:

      1. Enter a custom filter expression using Coveo query syntax.

      2. Click Apply.

  3. Click Save.

Delete a CPR model

Note

Models are automatically dissociated from all their associated query pipelines once they’re deleted.

  1. On the Models (platform-ca | platform-eu | platform-au) page, click the ML model that you want to delete, and then click More > Delete in the Action bar.

  2. In the Delete a model panel that appears, click Delete Model.

Review active model information

On the Models (platform-ca | platform-eu | platform-au) page, click the desired model (must be Active), and then click Open in the Action bar (see Reviewing model information).

Reference

Model embedding limits

The CPR model converts your content’s body text into numerical representations (vectors) in a process called embedding. It does this by breaking the text up into smaller segments called chunks, and each chunk is mapped as a distinct vector. For more information, see Embeddings.

Due to the amount of processing required for embeddings, the model is subject to the following embedding limits:

Note

The same chunking strategy is used for all sources and item types.

  • Up to 1 million items or 10 million chunks

    Notes

    • The maximum number of items depends on the item allocation of your product plan.

    • Coveo strongly recommends that you add a Semantic Encoder (SE) model as part of your CPR implementation. If you have more than one CPR model in your Coveo organization, for best results each CPR model should use only the items that are used by the SE model.

  • 1000 chunks per item

    Important

    This means that for a given item, there can be a maximum of 1000 chunks. So if an item is long with a lot of text, such as more than 200,000 words or 250 pages, the model will embed the item’s text until the 1000-chunk limit is reached. The remaining text won’t be embedded and therefore won’t be used by the model.

  • 250 words per chunk

    Note

    There can be an overlap of up to 10% between chunks. In other words, the last 10% of the previous chunk can be the first 10% of the next chunk.

"Status" column

On the Models (platform-ca | platform-eu | platform-au) page of the Administration Console, the Status column indicates the current state of your Coveo ML models.

The following table lists the possible model statuses and their definitions:

Status Definition Status icon

Active

The model is active and available.

Active

Build in progress

The model is currently building.

Building

Inactive

The model isn’t ready to be queried, such as when a model was recently created or the organization is offline.
Click See more details for additional information (see Review model information).

Inactive

Limited

Build issues exist that may affect model performance.
Click See more details for additional information (see Review model information).

Limited

Soon to be archived

The model will soon be archived because it hasn’t been queried for an extended period of time.
Click Delete to remove the model.
Learn more about archived models.

Archive pending

Error

An error prevented the model from being built successfully.
If it’s a temporary system error, check back soon. Otherwise, click See more details for additional information (see Review model information).

Error

Archived

The model was archived because it hasn’t been queried for at least 30 days.
Click Delete to remove the model.
Learn more about archived models.

Archived

Required privileges

By default, members with the required privileges can view and edit elements of the Models (platform-ca | platform-eu | platform-au) page.

The following table indicates the privileges required for members to manage Coveo Generic models (see Manage privileges and Privilege reference).

Action Service - Domain Required access level

View models

Machine Learning - Models
Organization - Organization
Search - Query pipelines

View

Manage models

Organization - Organization
Search - Query pipelines

View

Machine Learning - Models

Edit

Machine Learning - Allow content preview

Enable

Content - Sources

View All

Content - Fields

View

What’s next?

Associate the CPR model with a query pipeline.