About Coveo’s indexing capacity management

This is for:

Developer System Administrator

To protect its platform and ensure that your content is always processed properly, Coveo reserves the right to temporarily decline requests to index your data. This is called indexing capacity management.

What’s indexing capacity management?

Coveo’s capacity to index new content may occasionally be reduced by unforeseen circumstances. This can be due to you pushing a lot of data to Coveo within a short period or to technical issues. A reduced capacity results in longer processing delays for the data you send to Coveo for indexing.

In such a case, a protection mechanism is enabled to prevent tasks from piling up and overwhelming our systems. This mechanism may temporarily decline your requests to index items and/or security identities. This allows us to keep our platform stable and guarantee the processing of all your content.

Our capacity management mechanism only affects incoming requests to index content items and security identities. Once accepted, this data goes through the indexing pipeline at the same speed as usual. Similarly, queries on your search interfaces aren’t impacted.

Why does Coveo manage its indexing capacity?

When Coveo’s capacity to index content is reduced, managing incoming requests is necessary to ensure the stability and performance of the platform.

Letting indexing tasks pile up comes not only with unreasonable delays, but also with the risk of overwhelming our systems and jeopardizing the proper processing of your data.

Instead, we choose to have our mechanism temporarily decline certain calls and accept only the requests we can commit to process within an acceptable time frame. This ensures that all your data will be indexed properly.

When does Coveo manages its indexing capacity?

Our capacity management mechanism is automatically enabled when too many indexing tasks pile up for your organization. These requests may have been made by your sources and/or security providers, via the Push API or the Stream API.

Indexing capacity is monitored and managed on a per-organization basis. Requests made by other customers don’t impact our capacity to index your data.

Although we can’t predict the moments where our indexing capacity will be reduced, we strive to be transparent when it happens. You’ll receive an error message when using some of our API calls.

What happens when Coveo manages its indexing capacity?

The following is an example of what happens when Coveo’s capacity to index your content is lower than usual.

  1. You’ve made multiple calls to Coveo’s APIs to index content items and security identities. We’ve returned 200 OK responses, which indicate that we’ve committed to index this data. However, Coveo’s capacity to index your content is lower than usual, you’ve made your calls within a short period, faster than we can index your data.

  2. The indexing tasks are piling up and the delays are getting longer. Our indexing capacity mechanism is therefore enabled. As a result, your next API call receives a 429 Too Many Requests response with the following message: Several requests from your organization are still waiting to be processed, so we’ve temporarily stopped accepting any more. Try again in a few seconds. This means we’ve temporarily stopped accepting new indexing tasks to reduce delays and protect our systems.

  3. A few seconds later, you retry your API call. Since Coveo has had some time to process your previous requests, it can now accept new tasks from your organization. You therefore receive a 200 OK response, which means we’ve committed to process your data.

Which API calls are impacted by a reduced indexing capacity?

The calls that our mechanism may decline are dedicated to adding/removing items or security identities.

On the Push API, the following calls may be impacted by a reduced indexing capacity.

Calls affecting content items:

  • Add or update an item

  • Add, update, and/or delete a batch of items

  • Delete an item and optionally its children

  • Delete old items

Calls affecting security identities:

  • Add or update a security identity

  • Add, update, and/or delete a batch of security identities

  • Add or update an alias

  • Delete a security identity

  • Delete old security identities

Call affecting the source status:

  • Set the status of a Push source

On the Stream API, only the following calls are impacted:

  • Update a stream

  • Close a stream

How can I automatically handle the situation?

When your request is declined, you should retry it later to ensure that Coveo indexes your data. You can implement a retry mechanism in your code to avoid a manual process. We strongly suggest you use an exponential backoff retry strategy to ensure a minimal number of retries before a successful execution.

The following are examples of exponential backoff retry mechanisms in different programming languages.

Python 
import time
import requests

def call_api_with_retries(
    call_endpoint, call_data, max_nb_retries=7, initial_retry_delay_in_seconds=5, backoff_factor=3
):
    delay_in_seconds = initial_retry_delay_in_seconds
    nb_retries = 0
    while True:
        response = requests.post(call_endpoint, data=call_data)
        nb_retries += 1
        if response.status_code == 429 and nb_retries <= max_nb_retries:
            time.sleep(delay_in_seconds)
            delay_in_seconds = delay_in_seconds * backoff_factor
        else:
            response.raise_for_status()
            return response


if __name__ == "__main__":
    endpoint = "https://api.cloud.coveo.com/push/v1/organizations/mycoveoorg/providers/mysecprov/permissions"
    data = """
    {
        "identity": {
        "name": "AlphaTeam",
        "type": "GROUP"
     },
      "members": [
        {
          "name": "alice@example.com",
          "type": "USER"
        },
        {
          "name": "bob@example.com",
          "type": "USER"
        }
      ]
    }"""

    call_api_with_retries(call_endpoint=endpoint, call_data=data)
JavaScript 
function callAPIWithRetries(callAPIFunction, maxNbRetries = 7, initialRetryDelayInSeconds = 5, backoffFactor = 3) {
  let delayInMilliseconds = initialRetryDelayInSeconds * 1000;
  let nbRetries = 0;

  function retry() {
    return callAPIFunction().then(async (response) => {
      nbRetries++;
      if (response.status === 429 && nbRetries <= maxNbRetries) {
        await new Promise(r => setTimeout(r, delayInMilliseconds));
        delayInMilliseconds = delayInMilliseconds * backoffFactor;
        retry();
      } else {
        if (!response.ok) throw new Error(`HTTP error ${response.status} : ${response.statusText}`);
        return response;
      }
    });
  }
  return retry();
}

async function callPutPermissionsExample() {
	return await fetch('https://api.cloud.coveo.com/push/v1/organizations/mycoveoorg/providers/mysecprov/permissions',
      {
        method: 'PUT',
        body: JSON.stringify(
        {
          "identity": {
            "name": "AlphaTeam",
            "type": "GROUP"
          },
          "members": [
            {
              "name": "alice@example.com",
              "type": "USER"
            },
            {
              "name": "bob@example.com",
              "type": "USER"
            }
          ]
        })
    })
}

callAPIWithRetries(callPutPermissionsExample)
Java 11 
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;

public class CallApiWithRetries
{
    public static HttpResponse<String> callApiWithRetries(String callEndpoint,
                                                          String callData,
                                                          int maxNbRetries,
                                                          int initialRetryDelayInSeconds,
                                                          int backoffFactor)
            throws Exception
    {
        HttpClient client = HttpClient.newHttpClient();
        HttpRequest request = HttpRequest.newBuilder()
                                         .uri(new URI(callEndpoint))
                                         .PUT(HttpRequest.BodyPublishers.ofString(callData))
                                         .build();
        long delayInMilliseconds = initialRetryDelayInSeconds * 1000L;
        int nbRetries = 0;
        while (true) {
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
            nbRetries++;
            if (response.statusCode() == 429 && nbRetries <= maxNbRetries) {
                Thread.sleep(delayInMilliseconds);
                delayInMilliseconds = delayInMilliseconds * backoffFactor;
            } else {
                if (response.statusCode() >= 400) {
                    throw new Exception("HTTP error " + response.statusCode() + " : "
                                                      + response.body());
                }
                return response;
            }
        }
    }

    public static void main(String[] args) throws Exception
    {
        String endpoint = "https://api.cloud.coveo.com/push/v1/organizations/mycoveoorg/providers/mysecprov/permissions";
        String data = "{\"identity\": {\"name\": \"AlphaTeam\", \"type\": \"GROUP\" },\n"
                + "     \"members\": [{ \"name\": \"alice@example.com\", \"type\": \"USER\"  },\n"
                + "                   { \"name\": \"bob@example.com\",   \"type\": \"USER\"  }]}";
        HttpResponse<String> response = callApiWithRetries(endpoint, data, 7, 5, 3);
    }
}

How does indexing capacity relate to my organization limits?

A temporarily reduced indexing capacity and declined requests don’t affect your organization limits, and vice versa. These are two different concepts.

Your organization limits relate to what you do and consume in your organization. They apply at all times and are hard stops. For example, once you’ve reached your limit of items, you can’t index any more until you delete some.

Coveo accepts indexing requests based on the state of its system. Requests may exceptionally be declined to protect the platform in case of an unexpected issue or a sudden increase in the number of requests to process.