---
title: About Coveo Passage Retrieval (CPR)
slug: oaie5277
canonical_url: https://docs.coveo.com/en/oaie5277/
collection: leverage-machine-learning
source_format: adoc
---
# About Coveo Passage Retrieval (CPR)

> **Important**
>
> Passage Retrieval (CPR) is a paid product extension.
> Contact [Coveo Sales](https://www.coveo.com/en/contact) or your Account Manager to add CPR to your [organization](https://docs.coveo.com/en/185.md) license.

This article describes [what a Passage Retrieval model is](#what-is-a-passage-retrieval-model), [when to use it](#when-to-use-cpr), and how it uses [embeddings](#embeddings) and [content retrieval](#relevant-content-retrieval) to ground a large language model (LLM) to your enterprise content when generating answers in a [RAG](https://docs.coveo.com/en/p8ie0159.md) system.

## What is a Passage Retrieval model

A [Coveo Machine Learning (Coveo ML)](https://docs.coveo.com/en/188.md) Passage Retrieval (CPR) [model](https://docs.coveo.com/en/1012.md) retrieves the segments of text (passages) from your indexed enterprise content that are most relevant to a query.
As part of a [retrieval-augmented generation (RAG)](https://docs.coveo.com/en/p8ie0159.md) system, these passages are then provided to a generative large language model (LLM) to ground generated answers in your enterprise content.

The success of a [RAG](https://docs.coveo.com/en/p8ie0159.md) system depends on the quality and accessibility of its underlying content.
The challenge lies not only in retrieving content, but also in ensuring that it's up to date, contextually relevant, and secure.

CPR addresses these requirements by working with the [Coveo Platform](https://docs.coveo.com/en/186.md)'s existing [indexing](https://docs.coveo.com/en/204.md), AI, personalization, machine learning, relevance, and security capabilities.
A CPR model retrieves passages only from the content you specify, which resides in a secure [index](https://docs.coveo.com/en/204.md) in your [Coveo organization](https://docs.coveo.com/en/185.md).

To learn when and why to use CPR in your Coveo workflow, see [When to use CPR](https://docs.coveo.com/en/oaie5277.md#when-to-use-cpr).

> **Tip**
>
> By default, content retrieval is supported only for English content.
> However, Coveo offers beta support for content retrieval in languages other than English.
> Learn more about [multilingual content retrieval and answer generation](https://docs.coveo.com/en/p5ne0024.md).

## When to use CPR

You must [implement a CPR model](https://docs.coveo.com/en/oaoe7068.md) when using one of the following Coveo workflows, which rely on it for passage retrieval:

* When [**generating answers with the Coveo Search Agent**](#using-cpr-with-the-coveo-search-agent):
Use [CPR](https://docs.coveo.com/en/oaie9196.md) to retrieve the passages that the [Coveo Search Agent](https://docs.coveo.com/en/q2pe0294.md) uses to generate answers in your Coveo-powered search interface.

* When [**powering a custom RAG application**](#using-cpr-with-the-passage-retrieval-api):
Use [CPR](https://docs.coveo.com/en/oaie9196.md) through the [Passage Retrieval API](https://docs.coveo.com/en/o86c8334.md) to retrieve passages that you provide as input to your enterprise's own LLM-powered application.

> **Note**
>
> A [Passage Retrieval (CPR)](https://docs.coveo.com/en/oaie5277.md) implementation must include both a [CPR](https://docs.coveo.com/en/oaie9196.md) [model](https://docs.coveo.com/en/1012.md) and a [Semantic Encoder (SE)](https://docs.coveo.com/en/nbtb0041.md) [model](https://docs.coveo.com/en/1012.md).
> Both the CPR and SE models must be configured to use the same content.

### Using CPR with the Coveo Search Agent

This section provides a high-level look at how CPR retrieves passages when used with a [Coveo Search Agent](https://docs.coveo.com/en/q2pe0294.md) to generate answers in a Coveo-powered search interface.

![Passage retrieval flow | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/cpr-search-agent-flow.png)

1. When a user enters a query in a Coveo-powered search interface [configured for a Search Agent](https://docs.coveo.com/en/q32d5507.md), the Search Agent initiates the answer generation process by first calling the Search API to retrieve the most relevant [items](https://docs.coveo.com/en/210.md) in the [index](https://docs.coveo.com/en/204.md).
This is referred to as [first-stage content retrieval](#first-stage-content-retrieval).
+
As it normally does, the query passes through a [query pipeline](https://docs.coveo.com/en/180.md) where pipeline rules and machine learning are applied to optimize relevance.
However, the [Semantic Encoder (SE)](https://docs.coveo.com/en/nbtb0041.md) [model](https://docs.coveo.com/en/1012.md) adds vector search capabilities to your search interface in addition to the traditional lexical ([keyword](https://docs.coveo.com/en/2738.md)) search.
Vector search improves search results by using [embeddings](#embeddings) to retrieve the [items](https://docs.coveo.com/en/210.md) in the [index](https://docs.coveo.com/en/204.md) with high semantic similarity with the query.

1. The most relevant items retrieved during first-stage content retrieval are sent to the CPR model for [second-stage content retrieval](#second-stage-content-retrieval) where the CPR model retrieves the most relevant passages from the most relevant items.

1. The Search Agent calls the answer generation model and provides it with the most relevant passages obtained during second-stage content retrieval.
The answer generation model creates a prompt that includes instructions, the query, and the most relevant passages.

1. The prompt that's created by the answer generation model is sent to a third-party foundation model service where the LLM generates the answer based only on the grounded prompt.

1. The generated answer is returned to the Search Agent and displayed in the search interface.

### Using CPR with the Passage Retrieval API

This section provides a high-level look at how CPR retrieves passages when used with the Passage Retrieval API for use in your enterprise's own LLM-powered applications.

![Passage retrieval flow | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/cpr-overview.png)

1. A user enters a query in an LLM-powered application [configured for CPR](https://docs.coveo.com/en/oaoe7068.md).

1. As it normally does, the query passes through a [query pipeline](https://docs.coveo.com/en/180.md) where pipeline rules and machine learning are applied to optimize relevance.
However, the [Semantic Encoder (SE)](https://docs.coveo.com/en/nbtb0041.md) [model](https://docs.coveo.com/en/1012.md) adds vector search capabilities to the LLM-powered application in addition to the traditional lexical ([keyword](https://docs.coveo.com/en/2738.md)) search.
Vector search improves search results by using [embeddings](#embeddings) to retrieve the [items](https://docs.coveo.com/en/210.md) in the [index](https://docs.coveo.com/en/204.md) with high semantic similarity with the query.

1. The search engine identifies the most relevant [items](https://docs.coveo.com/en/210.md) in the [index](https://docs.coveo.com/en/204.md), and sends a list of the items to the CPR [model](https://docs.coveo.com/en/1012.md).
This is referred to as [first-stage content retrieval](#first-stage-content-retrieval).

1. The CPR [model](https://docs.coveo.com/en/1012.md) uses its [embeddings](#embeddings) to retrieve the most relevant passages from the items identified during first-stage content retrieval.
This is referred to as [second-stage content retrieval](#second-stage-content-retrieval).

1. The LLM-powered application [leverages the PR API](https://docs.coveo.com/en/o86c8334.md) to retrieve the passages identified by the CPR [model](https://docs.coveo.com/en/1012.md).

1. The LLM generates an output based on the passages retrieved from the PR API and returns it to the application for display to the user.

## CPR processes

This section examines the two main processes involved in passage retrieval with [CPR](https://docs.coveo.com/en/oaie9196.md):

* [Embeddings](#embeddings)

* [Relevant content retrieval](#relevant-content-retrieval)

### Embeddings

Embedding is a machine learning technique used in [natural language processing (NLP)](https://docs.coveo.com/en/ncc80129.md) to convert text data into numerical representations ([vectors](https://docs.coveo.com/en/nccf9008.md)) mapped to a vector space.
The vector space can represent the meaning of words, phrases, or documents as multidimensional points (vectors).
Therefore, vectors with similar meaning occupy relatively close positions within the vector space.
Embeddings are at the core of vector-based search, such as semantic search, that's used to find similarities based on meaning and context.

The following is a graphical representation of a vector space with each dot representing a vector (embedding).
Each vector is mapped to a specific position in the multi-dimensional space.
Vectors with similar meaning occupy relatively close positions.

![Vector space | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/rga-cpr-vector-space.png)

A CPR implementation must include both a [Passage Retrieval (CPR) model](https://docs.coveo.com/en/oaie5476.md) and a [Semantic Encoder (SE) model](https://docs.coveo.com/en/nb6a0483.md).
Each model creates and uses its own embedding vector space.
The models have the dual purpose of creating the embeddings, and then referencing their respective vector spaces at query time to retrieve relevant content.
The SE model uses the embeddings for [first-stage content retrieval](#first-stage-content-retrieval), and the CPR model uses the embeddings for [second-stage content retrieval](#second-stage-content-retrieval).

The CPR and SE models use a pre-trained sentence transformer language model to create the embeddings using [chunks](#chunking).
Each [model](https://docs.coveo.com/en/1012.md) creates the embeddings when the model builds, and then updates the embeddings based on their respective build schedules.
The CPR model stores the embeddings in model memory, while the SE model stores the embeddings in the Coveo unified [index](https://docs.coveo.com/en/204.md).
For more information, see [What does a CPR model do?](https://docs.coveo.com/en/oaie5476.md#what-does-a-cpr-model-do) and [What does an SE model do?](https://docs.coveo.com/en/nb890247.md#what-does-an-se-model-do).

#### Chunking

Chunking is a processing strategy that's used to break down large pieces of text into smaller segments called _chunks_ (passages).
Large language models (LLMs) then use these chunks to create the [embeddings](https://docs.coveo.com/en/ncc87383.md).
Each chunk is mapped as a distinct vector in the embedding vector space.

CPR and SE models use optimized chunking strategies to ensure that a passage contains just enough text and context to be semantically relevant.

When a CPR or SE model builds, the content of each item that's used by the model is broken down into passages that are then mapped as individual vectors.
An effective chunking strategy ensures that when it comes time to [retrieve content](#relevant-content-retrieval), the models can find the most contextually relevant items based on the query.

> **Note**
>
> You can [choose the chunking strategy](https://docs.coveo.com/en/oaie5476.md#set-the-chunking-strategy) that the [CPR](https://docs.coveo.com/en/oaie9196.md) [model](https://docs.coveo.com/en/1012.md) uses to create the chunks.

![Vector space | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/cpr-chunk-example.png)

> **Note**
>
> CPR and SE models are subject to chunking constraints.
> For more information, see the embedding constraints for [CPR](https://docs.coveo.com/en/oaod5329.md#model-embedding-constraints) and [SE](https://docs.coveo.com/en/nbo90598.md#model-embedding-constraints).

### Relevant content retrieval

When using generative AI to generate text from data, it's essential to identify and control the content that will be used as the data.

To ensure that CPR retrieves content based on the most relevant data, the process involves two layers of content retrieval.

> **Note**
>
> Your enterprise content permissions are enforced during content retrieval.
> This ensures that an output generated from the retrieved content only shows the information that the authenticated user is allowed to access.
> For more information, see [Search Agent data security](https://docs.coveo.com/en/q3qb0518.md).

1. [First-stage content retrieval](#first-stage-content-retrieval) identifies the most relevant [items](https://docs.coveo.com/en/210.md) in the [index](https://docs.coveo.com/en/204.md).

1. [Second-stage content retrieval](#second-stage-content-retrieval) identifies the most relevant passages from the items retrieved during first-stage content retrieval.

![CPR content retrieval | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/cpr-content-retrieval.png)

> **Tip**
>
> By default, content retrieval is supported only for English content.
> However, Coveo offers beta support for content retrieval in languages other than English.
> Learn more about [multilingual content retrieval and answer generation](https://docs.coveo.com/en/p5ne0024.md).

#### First-stage content retrieval

The initial content retrieval occurs at the [item](https://docs.coveo.com/en/210.md) level, where the Coveo search engine finds the most relevant [items](https://docs.coveo.com/en/210.md) in the [index](https://docs.coveo.com/en/204.md) for a given [query](https://docs.coveo.com/en/231.md).
When a user performs a query in an application that leverages CPR, the query passes through a [query pipeline](https://docs.coveo.com/en/180.md) where [rules](https://docs.coveo.com/en/236.md) and machine learning are applied to optimize relevance.
When first-stage content retrieval is complete, the most relevant [items](https://docs.coveo.com/en/210.md) are sent to the CPR [model](https://docs.coveo.com/en/1012.md).
The CPR [model](https://docs.coveo.com/en/1012.md) takes only these most relevant items into consideration during [second-stage content retrieval](#second-stage-content-retrieval).

Because the CPR model relies on the effectiveness of first-stage content retrieval, a [Semantic Encoder (SE) model](https://docs.coveo.com/en/nb6a0483.md) ensures that the most relevant items are available for CPR.
Given that user queries for passage retrieval are typically complex and use natural language, retrieving passages based on lexical search results alone wouldn't necessarily provide contextually relevant passages.
An SE model uses vector-based search to extract the meaning in a complex natural language query.
As shown in the following diagram, for a given query, the SE model embeds the query into the vector space in the index to find the items with high semantic similarity with the query.
This means that search results are based on the meaning and context of the query, and not just on keywords.
For more information, see [About Semantic Encoder (SE)](https://docs.coveo.com/en/nb6a0483.md).

![Vector space query | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/grouped-se.png)

#### Second-stage content retrieval

With [first-stage content retrieval](#first-stage-content-retrieval) complete, the CPR [model](https://docs.coveo.com/en/1012.md) now knows the most contextually relevant [items](https://docs.coveo.com/en/210.md) in the [index](https://docs.coveo.com/en/204.md) to which the user has access.
The next step is to identify the most relevant passages ([chunks](#chunking)) from those items.
The passages will be used as the data from which the response will be generated.

The CPR [model](https://docs.coveo.com/en/1012.md) uses the [embeddings](#embeddings) vector space in its memory to find the most relevant passages from the items identified during first-stage retrieval.
As shown in the following diagram, for a given query the CPR [model](https://docs.coveo.com/en/1012.md) embeds the query into the vector space and performs a vector search to find the passages with the highest semantic similarity with the query.

![Vector space query | Coveo](https://docs.coveo.com/en/assets/images/leverage-machine-learning/grouped-cpr.png)

Second-stage content retrieval results in a list of the most relevant passages from the most relevant [items](https://docs.coveo.com/en/210.md) in the [index](https://docs.coveo.com/en/204.md).

However, prior to finalizing the list of most relevant passages, the [CPR](https://docs.coveo.com/en/oaie9196.md) model applies the following additional processes:

* [**Score fusion**](#score-fusion): Used to adjust the ranking of the passages by taking into account the relevance of their source [item](https://docs.coveo.com/en/210.md), which was retrieved during [first-stage content retrieval](#first-stage-content-retrieval).

  > **Note**
  >
  > Score fusion is enabled by default for all newly created [CPR](https://docs.coveo.com/en/oaie9196.md) [models](https://docs.coveo.com/en/1012.md), and for all existing [CPR](https://docs.coveo.com/en/oaie9196.md) [models](https://docs.coveo.com/en/1012.md) linked to a [Coveo Search Agent](https://docs.coveo.com/en/q2pe0294.md).
  > For existing [CPR](https://docs.coveo.com/en/oaie9196.md) [models](https://docs.coveo.com/en/1012.md) that are used with the Passage Retrieval API (PR API) to provide passages to an enterprise's own LLM-powered applications, [contact Coveo Support](https://connect.coveo.com/s/case/Case/Default) to enable score fusion for your [model](https://docs.coveo.com/en/1012.md).

* [**Passage merging**](#passage-merging): Used to merge passages that are from the same [item](https://docs.coveo.com/en/210.md) and that are in close proximity to each other.
This is done to reduce redundancy among passages and improve passage coherence.
+
> **Note**
>
> Only [CPR](https://docs.coveo.com/en/oaie9196.md) [models](https://docs.coveo.com/en/1012.md) that use [structure-aware chunking](https://docs.coveo.com/en/p9ub0044.md) implement passage merging.

What results is a final list of passages ranked and combined to form a more relevant and cohesive set of passages.
The passages are now available to be used for answer generation when using the [Coveo Search Agent](https://docs.coveo.com/en/q2pe0294.md) or [Passage Retrieval API](https://docs.coveo.com/en/o86c8334.md).

> **Note**
>
> You can set a custom value for the maximum number of items that the CPR model considers when retrieving the most relevant passages.
> Though the default value usually provides good results, you can set a custom value if needed.
>
> When using the [CPR](https://docs.coveo.com/en/oaie9196.md) model with the Coveo Search Agent, you can set a custom value in the [Search Agent configuration](https://docs.coveo.com/en/q2qg1117.md#create-a-search-agent).
> When using the [CPR](https://docs.coveo.com/en/oaie9196.md) model with the Passage Retrieval API, you can set a custom value for the [model association configuration in the query pipeline](https://docs.coveo.com/en/oaie6403.md#cpr-model-association-advanced-configuration).

#### Score fusion

After the [CPR](https://docs.coveo.com/en/oaie9196.md) [model](https://docs.coveo.com/en/1012.md) retrieves the most relevant passages, it applies a process called _score fusion_ to refine the ranking of the retrieved passages based on the relevance of their source [items](https://docs.coveo.com/en/210.md).
The model gives greater weight to passages originating from items that received higher relevance scores during [first-stage content retrieval](#first-stage-content-retrieval).

First-stage content retrieval identifies the most relevant items using lexical, semantic, behavioral, and business-rule signals.
Ignoring these signals would discard important information not captured by passage retrieval, which relies only on semantic similarity.
As a result, relevant passages from relevant indexed items could be missed, making results less aligned with overall search relevance.
Incorporating item-level relevance scores results in a more accurate and robust passage ranking that better aligns with the most relevant items.

> **Note**
>
> Score fusion is enabled by default for all newly created [CPR](https://docs.coveo.com/en/oaie9196.md) [models](https://docs.coveo.com/en/1012.md), and for all existing [CPR](https://docs.coveo.com/en/oaie9196.md) [models](https://docs.coveo.com/en/1012.md) linked to a [Coveo Search Agent](https://docs.coveo.com/en/q2pe0294.md).
> For existing [CPR](https://docs.coveo.com/en/oaie9196.md) [models](https://docs.coveo.com/en/1012.md) that are used with the Passage Retrieval API (PR API) to provide passages to an enterprise's own LLM-powered applications, [contact Coveo Support](https://connect.coveo.com/s/case/Case/Default) to enable score fusion for your [model](https://docs.coveo.com/en/1012.md).

To adjust passage ranking based on source item relevance, the model uses Reciprocal Rank Fusion (RRF).
RRF is an algorithm commonly used in [RAG](https://docs.coveo.com/en/p8ie0159.md) systems to combine multiple ranked lists into a single unified ranking.
At a high level, RRF gives more importance to elements that rank highly across multiple lists.
Since item retrieval and passage retrieval are performed by different models with distinct relevance scoring methods, the RRF algorithm is used to combine their rankings into a single passage ranking that better reflects overall relevance.

**Example**

Given the following ranked items after first-stage item retrieval:

| Relevance ranking | Item title |
|---|---|
| 1st | About RAG systems |
| 2nd | How RAG works |
| 3rd | RAG system design |
| 4th | Better RAG search |
| 5th | RAG Examples |

And given the following ranked passages after second-stage passage retrieval:

| Relevance ranking | Passage | Relevance score | Source item (ranking in first-stage retrieval) |
|---|---|---|---|
| 1st | Passage 1 | 0.891 | About RAG systems (1st) |
| 2nd | Passage 2 | 0.871 | RAG Workflow (not retrieved during first-stage retrieval) |
| 3rd | Passage 3 | 0.868 | How RAG works (2nd) |
| 4th | Passage 4 | 0.855 | Better RAG search (4th) |
| 5th | Passage 5 | 0.844 | RAG system design (3rd) |
| 6th | Passage 6 | 0.841 | RAG Methods (not retrieved during first-stage retrieval) |

After applying score fusion, the final ranking of the passages is as follows:

| Relevance ranking | Passage | RRF score | Source item (ranking in first-stage retrieval) |
|---|---|---|---|
| 1st | Passage 1 | 0.200 | About RAG systems (1st) |
| 2nd | Passage 3 | 0.167 | How RAG works (2nd) |
| 3rd | Passage 5 | 0.143 | RAG system design (3rd) |
| 4th | Passage 4 | 0.125 | Better RAG search (4th) |
| 5th | Passage 2 | 0.071 | RAG Workflow (not retrieved during first-stage retrieval) |
| 6th | Passage 6 | 0.063 | RAG Methods (not retrieved during first-stage retrieval) |

The score values in this example are for illustrative purposes only and don't reflect actual RRF scores relative to relevance scores.
#### Passage merging

After the [CPR](https://docs.coveo.com/en/oaie9196.md) [model](https://docs.coveo.com/en/1012.md) [retrieves the most relevant passages](#second-stage-content-retrieval), and after applying [score fusion](#score-fusion), the model applies _passage merging_ to the passages.

> **Note**
>
> Only [CPR](https://docs.coveo.com/en/oaie9196.md) [models](https://docs.coveo.com/en/1012.md) that use [structure-aware chunking](https://docs.coveo.com/en/p9ub0044.md) implement passage merging.

The [model](https://docs.coveo.com/en/1012.md) evaluates the retrieved passages to identify those that are from the same source [item](https://docs.coveo.com/en/210.md) and that overlap, are next to each other, or in close proximity to each other, in the original content.
When such passages are found, the model merges them to create a single, more contextually complete passage.
This results in more coherent passages that provide more context and reduce redundancy between passages, making it easier for a generative LLM to understand when generating a response.
Merged passages retain the relevance score of the highest-ranked passage among those merged, ensuring that the most relevant information is preserved in the final passage ranking.

> **Notes**
>
> * A merged passage can contain up to 10 separate passages.
> If more than 10 passages from the same item are eligible for merging, the model merges the first 10 passages into one, and then merges the remaining passages into a second passage.
>
> * Merging passages can increase the average number of tokens per passage by up to 25%, and the maximum by up to 200%, which can impact downstream LLM applications.

## What's next?

[Implement CPR](https://docs.coveo.com/en/oaoe7068.md) in your Coveo organization.