About the search process

This is for:

In this article

Search process overview
How queries are built
Authenticating
What’s next?

Each time a user triggers a query in a search interface, that query must go through a whole process before the results can be displayed back to the user. Having a high-level understanding of the search process will help you design your search experience.

Search process overview

The following diagram and accompanying explanations provide a fairly high-level overview of a typical search scenario.

The user types some text in the search box and submits a new query.
The search interface, typically relying on the Coveo Atomic library, builds the query by combining the search keywords submitted by the user with other generated filter expressions and parameters.
The search interface sends the query to the Search API.
The Search API parses the query and determines the appropriate query pipeline.
The Search API routes the query to the appropriate query pipeline.
The query pipeline applies various rules to modify the query depending on its context.
The query pipeline forwards the modified query to the index.
The index executes the query.
The index returns the ranked query response to the Search API.
The Search API builds the JSON query response (for example, populates the results array, evaluates the query duration, etc.).
The Search API sends the query response to the search interface.
The search interface processes the query response and notifies its components (for example, result list, facet, etc.) that the query was successful.
Affected components update themselves (for example, the result list renders the query results, facets adjust their values and numbers, etc.).

How queries are built

As you may have inferred from the second step of the above diagram and explanations, a query typically contains far more than the keywords entered by the user (i.e, the basic query expression (q)). This means that queries can be built differently depending on the context, to yield more personalized and relevant results to different users, even when they enter the same basic query expression.

The Atomic library builds the basic query expression entered in the atomic-search-box component. It also builds other types of expressions based on components such as facets and sorting. In a standard Coveo-powered search implementation, a developer could complete a full project without ever having to build an expression manually.

About query expressions

In a Coveo-powered search implementation, the query that’s sent to the index is made up of several discrete query expressions which can be just as important as the user input. Besides the basic query expression, these can include the following:

advanced query expression (aq): generated when toggling facet values
constant query expression (cq): often enforced to confine the scope of a search for a specific search interface

Note

nested queries should never be in the constant query expression, as they prevent caching and will slow down all queries.
large query expression (lq): typically added when using a case deflection-like search interface
disjunction query expression (dq): may be added in the query pipeline by an Automatic Relevance Tuning (ART) model

The end query that’s evaluated against the index is essentially (((q && aq) || dq) && cq). In plain words, the index returns results which match cq, and either both of q and aq, or just dq. Additionally, lq is processed by the Intelligent Term Detection (ITD) feature to inject relevant keywords in q.

Example

A customer searches for a snowboard on a commerce search page, so the q part of the combined query expression is snowboard.

The customer interacts with the Price facet to select prices under $300, so the aq is @price<=300.

The ART model on the assigned query pipeline has learned that customers were often interested in purchasing a particular snowboard (for example, result item) with a permanentid value of z0f0rcprlia27, so the dq is @permanentid=z0f0rcprlia27.

This results in the following combined query expression:

((snowboard AND @price<=300) OR @permanentid=z0f0rcprlia27)

Notes

All query expressions rely on the Coveo query syntax. We recommend that you understand this syntax before building your own set of expressions.
If you require more customization, there are various non-expression parameters that can be applied directly through the Search API when creating a query.

Using query expressions to filter at query time

A common use of expressions is filtering at query time. This is a recommended strategy to reduce the scope of certain search pages or search driven listing pages.

Example

A support agent authenticates on a customer service search interface.

Their generated search token enforces the filter NOT @source=="Marketing" to ensure that marketing items don’t appear in their query results. The constant query expression in every query sent from that interface is NOT (@source=="Marketing"). The Search API only returns results which have the right @source field.

Notes

The index must be clean of unwanted items before filtering at query time. You can also filter at indexing time.
While you can define filter expressions directly in the code of your search interface, a more flexible approach is to create query pipeline filter rules. You’ll learn more about query pipelines in the Tune relevance section of the Coveo implementation guide.

Authenticating

An index can contain publicly accessible items as well as items that only specific users can see, in accordance with the original repository permission system. When querying the index, the Search API only returns the result items that the user is allowed to access. It’s therefore important that you generate the proper access token for your users.

In a search interface that only displays public content, the access token can just consist of a public API key. However, a search token must be generated server-side in a search interface that displays items according to a permission model. This is done using an API key with the privilege to impersonate users (that is, the Allowed access level on the Impersonate domain), which should never be exposed publicly. Use the appropriate method to add authentication to your search page.

Note

Even in a non-secured search page, you may want to consider using the search token authentication, as search tokens can enforce a filter expression, search hub, and query pipeline.

What’s next?

The Leverage the Coveo Atomic library article explains how to implement your search interfaces using the Atomic library.