About Coveo Passage Retrieval (CPR)

Chunking is a processing strategy that’s used to break down large pieces of text into smaller segments called chunks (passages). Large language models (LLMs) then use these chunks to create the embeddings. Each chunk is mapped as a distinct vector in the embedding vector space.

CPR and SE models use optimized chunking strategies to ensure that a passage contains just enough text and context to be semantically relevant.

When a CPR or SE model builds, the content of each item that’s used by the model is broken down into passages that are then mapped as individual vectors. An effective chunking strategy ensures that when it comes time to retrieve content, the models can find the most contextually relevant items based on the query.

The initial content retrieval occurs at the item level, where the Coveo search engine finds the most relevant items in the index for a given query. When a user performs a query in an application that leverages CPR, the query passes through a query pipeline where rules and machine learning are applied to optimize relevance. When first-stage content retrieval is complete, the most relevant items are sent to the CPR model. The CPR model takes only these most relevant items into consideration during second-stage content retrieval.

Because the CPR model relies on the effectiveness of first-stage content retrieval, a Semantic Encoder (SE) model ensures that the most relevant items are available for CPR. Given that user queries for passage retrieval are typically complex and use natural language, retrieving passages based on lexical search results alone wouldn’t necessarily provide contextually relevant passages. An SE model uses vector-based search to extract the meaning in a complex natural language query. As shown in the following diagram, for a given query, the SE model embeds the query into the vector space in the index to find the items with high semantic similarity with the query. This means that search results are based on the meaning and context of the query, and not just on keywords. For more information, see About Semantic Encoder (SE).

With first-stage content retrieval complete, the CPR model now knows the most contextually relevant items in the index to which the user has access. The next step is to identify the most relevant passages (chunks) from those items. The passages will be used as the data from which the response will be generated.

The CPR model uses the embeddings vector space in its memory to find the most relevant passages from the items identified during first-stage retrieval. As shown in the following diagram, for a given query the CPR model embeds the query into the vector space and performs a vector search to find the passages with the highest semantic similarity with the query.

Second-stage content retrieval results in a list of the most relevant passages from the most relevant items in the index.

However, prior to finalizing the list of most relevant passages, the CPR model applies the following additional processes:

What results is a final list of passages ranked and combined to form a more relevant and cohesive set of passages. The passages are now available to be used for answer generation when using the Coveo Search Agent or Passage Retrieval API.

After the CPR model retrieves the most relevant passages, it applies a process called score fusion to refine the ranking of the retrieved passages based on the relevance of their source items. The model gives greater weight to passages originating from items that received higher relevance scores during first-stage content retrieval.

First-stage content retrieval identifies the most relevant items using lexical, semantic, behavioral, and business-rule signals. Ignoring these signals would discard important information not captured by passage retrieval, which relies only on semantic similarity. As a result, relevant passages from relevant indexed items could be missed, making results less aligned with overall search relevance. Incorporating item-level relevance scores results in a more accurate and robust passage ranking that better aligns with the most relevant items.

To adjust passage ranking based on source item relevance, the model uses Reciprocal Rank Fusion (RRF). RRF is an algorithm commonly used in RAG systems to combine multiple ranked lists into a single unified ranking. At a high level, RRF gives more importance to elements that rank highly across multiple lists. Since item retrieval and passage retrieval are performed by different models with distinct relevance scoring methods, the RRF algorithm is used to combine their rankings into a single passage ranking that better reflects overall relevance.

After the CPR model retrieves the most relevant passages, and after applying score fusion, the model applies passage merging to the passages.

The model evaluates the retrieved passages to identify those that are from the same source item and that overlap, are next to each other, or in close proximity to each other, in the original content. When such passages are found, the model merges them to create a single, more contextually complete passage. This results in more coherent passages that provide more context and reduce redundancy between passages, making it easier for a generative LLM to understand when generating a response. Merged passages retain the relevance score of the highest-ranked passage among those merged, ensuring that the most relevant information is preserved in the final passage ranking.