Number of workers

Workers are the Coveo Crawling Module components responsible for executing update tasks requested by Coveo. For more information on these operations, see Refresh, rescan, and rebuild.

There are two types of workers:

Once you have deployed the Crawling Module, you can edit its configuration to create or delete workers. You can also monitor your workers from the Crawling Module component dashboard.

Content workers

A worker executes one content update operation at a time. When you have more than one content worker, each works on its own task. For example, with two content workers, a Jira Software source refresh and a File System source rescan can happen simultaneously, while a single content worker would execute these tasks one after the other.

When all content workers are busy, any new content update task is delayed until one of them becomes available. Content update operations repeatedly executed behind schedule indicate that you have too few content workers. Consequently, the search results in your Coveo-powered search interface may not reflect your actual data.

By default, after its deployment, the Crawling Module has four content workers.

Tip
Leading practice

As a rule of thumb, we recommend starting with one content worker between two sources associated with this Crawling Module. So, if you have four Crawling Module sources, two content workers should typically suffice.

Security workers

Each Crawling Module source that indexes permissions created with the new connector version comes with at least one security identity provider that provides Coveo with the permission model of each retrieved item. Coveo can then replicate this model in the search interfaces it powers so that each end user can see only the content they’re allowed to access in your original repository. The Security Identities (platform-ca | platform-eu | platform-au) Administration Console page shows a list of your security identity providers.

Security workers are responsible for crawling the content permissions and feeding security identity providers this information. They also extract members from group security identities and associate users with their email identity. For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

Similarly to a content worker, a security worker can execute only one update operation at a time. It’s therefore crucial to have an adequate number to ensure that Coveo search results reflect your actual content permissions.

By default, after its deployment, the Crawling Module has four security workers.

Tip
Leading practice

As a rule of thumb, we recommend starting with one security worker between two sources that index permissions associated with this Crawling Module. These are sources that have the Same users and groups as in your content system content security option selected, or "sourceVisibility": "SECURED" if the source was created via API.

So, if you have four Crawling Module sources, and only two of them index permissions, a single security worker should typically suffice.

What’s next?