Understanding the Architecture of Coveo for Sitecore

This page covers the architecture of Coveo for Sitecore. You will learn about its different parts and how they interact with each other. This can be extremely valuable if you need to troubleshoot issues.

The Building Blocks

From a conceptual point of view, Coveo for Sitecore is made of the following building blocks.

Block 1 - Coveo Search Provider

This module is deployed as a Sitecore package (ZIP file). The Coveo Search Provider acts as a bridge between Sitecore and the Coveo Cloud Platform; it is responsible for handling queries ran from within Sitecore and for synchronizing your search indexes with the Coveo index.

Block 2 - Coveo Cloud Platform

This module is the service responsible for creating, managing, and maintaining your Coveo index.

Among other things, it contains:

  • a Push API, which is used to index your Sitecore items.
  • a Sitecore security provider, which handles Sitecore security permissions and allows you to find only what you are allowed to see when you perform a query.

Ultimately, the Coveo Cloud Platform is the module that handles all queries in the background.

Event Workflow

Depending on the type of action that you trigger in Coveo for Sitecore, a different sequence of events occur under the cover. Here are explained the most common scenarios.

Scenario 1 - Rebuilding Your Search Indexes

Rebuilding your search indexes causes these events to occur:

  1. The Sitecore application from where you launched the rebuild (Sitecore Desktop or Content Editor) tells the Search Provider to rebuild the chosen search index (e.g. Sitecore_master_index).
  2. The Coveo Search Provider tells the Coveo Cloud Platform to create everything necessary on the Coveo side to begin indexing Sitecore items. This includes:
    1. one Coveo field for each one of the Sitecore fields defined by Sitecore templates as well as for each computed field defined in Coveo.SearchProvider.config.
    2. a field mapping for each one of the Coveo field to map it to the corresponding Sitecore field.
    3. a source of type Sitecore (e.g. Sitecore_master_index - <hostname>-<Sitecore instance name>).
    4. an Expanded Sitecore Security Provider (e.g. Expanded Sitecore Security Provider for <hostname>-<Sitecore instance name>). The Sitecore Security Provider expands security permissions found on Sitecore items.
  3. The Coveo Search Provider begins to fetch the users and roles (known as identities) from Sitecore.
  4. The Coveo Search Provider pushes those identities to the Push API. 
  5. The Coveo Search Provider begins to fetch Sitecore items from the database corresponding to the chosen search index (e.g. master).
  6. The Coveo Search Provider pushes those Sitecore items to the Push API.
  7. The Push API indexes those Sitecore items, including their metadata fields and their security permissions. 

Scenario 2 - Performing a Query from a Coveo Search Page

Performing a query from a Coveo Search Page causes these events to occur:

  1. The Coveo Search Page sends the query to the Coveo Search REST Endpoint.
    The Coveo Search REST endpoint forwards the query to the Coveo Cloud Platform Search Endpoint.
  2. Coveo executes the query and returns only the results that the user is allowed to see.
  3. The Coveo Cloud Platform returns search results to the Coveo Search REST Endpoint in a JSON format.
  4. The Coveo Search REST Endpoint returns the search results to the Coveo Search Page (i.e., JS UI Search Interface).

Scenario 3 - Performing a Query using the Content Search API

Performing a query from the Content Editor causes these events to occur:

  1. The Content Search API builds a LINQ-based expression representing the query entered by the user and forwards it to the Coveo Search Provider.
  2. The Coveo Search Provider parses this expression and turns it into a query format understood by Coveo.
  3. The Coveo Search Provider forwards the query to the Coveo Search API Proxy.
  4. The Coveo Search API Proxy forwards the query to the Coveo Search Service. Coveo executes the query and returns only the results that the user is allowed to see.
  5. The Coveo Search Provider returns those search results in a format understood by Sitecore.

Block 1 - Coveo Search Provider

This module is deployed as a Sitecore package (ZIP file). The Coveo Search Provider acts as a bridge between Sitecore and Coveo Enterprise Search; it is responsible for handling queries ran from within Sitecore and for synchronizing your search indexes with the Coveo index.

Block 2 - Coveo Enterprise Search (CES)

This module is installed by the Coveo Enterprise Search installer. It is responsible for creating, managing, and maintaining your Coveo index.

Among other things, it spawns:

  • a Queue crawler, which continually retrieves Sitecore items from your RabbitMQ queues and indexes them
  • a Sitecore security provider, which handles Sitecore security permissions and allows you to find only what you are allowed to see when you perform a query

Ultimately, CES really is the module that handles all queries under the cover.

Block 3 - Coveo Admin Service

This module is installed by the Coveo Enterprise Search installer. It runs as a separate service from CES and serves as an automation tool for the Coveo index. It exposes a web service, and you can send it commands to automate all kinds of tasks, such as creating a source or a security provider, refreshing or deleting a source, creating or updating fields, etc.

Block 4 - RabbitMQ

This module is installed by the Coveo Enterprise Search installer. It is a messaging platform used jointly by the Coveo Search Provider and CES. Internally, RabbitMQ has a database into which it stores message queues. Sitecore items get pushed to those queues by the Coveo Search Provider, and in turn the Queue crawler retrieves those items and indexes them in the Coveo index.

Block 5 - Coveo Search API Service

This module is installed by the Coveo Search API installer. It runs as a separate service from CES and serves as a global REST endpoint that handles queries ran from within Sitecore. Once it receives a query, it forwards it to the Coveo Search Service, which is a SOAP-based web service exposed by CES. The Coveo Search Service executes the query and then returns the results as a SOAP response to the Search API module, which itself returns results as a REST response to whichever Sitecore module initially triggered the query.

Event Workflow

Depending on the type of action that you trigger in Coveo for Sitecore (be it a rebuild of your search indexes or a simple query), a different sequence of events occurs. Here are explained the most common scenarios.

Scenario 1 - Rebuilding Your Search Indexes

Rebuilding your search indexes causes these events to occur:

  1. The Sitecore application from where you launched the rebuild (Sitecore Desktop or Content Editor) tells the Search Provider to rebuild the chosen search index (e.g. Sitecore_master_index).
  2. The Coveo Search Provider then tells RabbitMQ to create a queue corresponding to the chosen search index (e.g. Sitecore_master_index - <hostname>-<Sitecore instance name>).
  3. The Coveo Search Provider tells the Coveo Admin Service to create everything necessary on the CES side to begin indexing Sitecore items. This includes:
    1. a Field Set which defines a CES custom field for each one of the Sitecore fields defined by Sitecore templates as well as for each computed field defined in Coveo.SearchProvider.config.
    2. a User Identity which contains the credentials of a Sitecore admnistrator (e.g. Sitecore Admin for <hostname>-<Sitecore instance name>).
    3. a source of type Queue (e.g. Sitecore_master_index - <hostname>-<Sitecore instance name>).
    4. a security provider of type Sitecore (e.g. Sitecore Security Provider for <hostname>-<Sitecore instance name>).
  4. The Coveo Search Provider begins to fetch Sitecore items from the database corresponding to the chosen search index (e.g. master).
  5. The Coveo Search Provider pushes those Sitecore items to the RabbitMQ queue corresponding to the chosen search index (e.g. Sitecore_master_index - <hostname>-<Sitecore instance name>).
  6. The Queue crawler begins to fetch Sitecore items stored in the RabbitMQ queue.
  7. The Queue crawler indexes those Sitecore items including their metadata fields as well as their security permissions.
  8. The Sitecore Security Provider expands security permissions found on Sitecore items in order to replicate the security model of Sitecore. Thus, a query only returns the items that a user is allowed to see.

Scenario 2 - Performing a Query from the Content Search API

Queries coded and executed against the ContentSearch API causes these events to occur:

  1. The Coveo Search Provider parses this expression and turns it into a query format understood by CES.
  2. The Coveo Search Provider forwards the query to the Coveo Search API Proxy.
  3. The Coveo Search API Proxy forwards the query to the Coveo Search API Service.
  4. The Coveo Search API Service tells the Coveo index to execute the query.
  5. CES executes the query and returns only the results that the user is allowed to see.
  6. The Coveo Search Service returns search results to the Coveo Search Provider.
  7. The Coveo Search Provider returns those search results in a format understood by Sitecore.

Scenario 3 - Performing a Query from a Coveo Search Page

Performing a query from a Coveo Search Page causes these events to occur:

  1. The Coveo Search Page sends the query to the Coveo Search REST Endpoint.
  2. The Coveo Search REST endpoint forwards the query to the Coveo Search API Service
  3. The Coveo Search API Service forwards the query to the Coveo Search Service.
  4. CES executes the query and returns only the results that the user is allowed to see.
  5. The Coveo Search Service returns search results to the Coveo Search API Service in a SOAP format.
  6. The Coveo Search API Service returns search results to the Coveo Search REST Endpoint in a JSON format.
  7. The Coveo Search REST Endpoint returns the search results to the Coveo Search Page (i.e., JS UI Search Interface).