How does Coveo ensure efficient semantic retrieval?

When content is indexed, Coveo encodes it into vector representations, called embeddings. For text items, Coveo divides each item into passages and encodes each passage as a separate vector, so one item can have multiple vectors. For commerce catalogs, Coveo encodes each product as a product vector. At query time, Coveo also encodes the query as a vector. The index then calculates the similarity between the query vector and the stored vectors to rank the closest passages, items, or products by semantic similarity.

To perform this retrieval efficiently, Coveo uses HNSW, a graph-based approximate nearest-neighbor algorithm that finds the stored vectors most similar to the query vector.

Quantization is the process of representing numeric values at lower precision so that they occupy less memory. By default, vector components are represented as 32-bit floating-point values. At scale, these vectors can require significant memory. To reduce this, the Coveo index uses built-in quantization to represent vector components with 16-bit floating-point values instead. This shrinks the memory footprint, supports more vectors, and accelerates vector-based calculations. Reducing each component from 32 bits to 16 bits decreases the memory required for vector data by approximately 50%.

The HNSW index continues to perform vector-based similarity calculations on this compact representation. Because 16-bit floating-point retains most of each component’s precision, the impact on retrieval accuracy is minimal. Coveo therefore uses 16-bit quantization as a balance between memory savings and semantic-search quality.

Learn how Coveo applies semantic retrieval in the Semantic Encoder (SE) model for search and the Catalog Semantic Encoder (CSE) model for commerce.