Push API quotas and constraints
Push API quotas and constraints
The Push API enforces certain constraints on request size, request frequency, on items and on security identities.
You can review your Push API usage on the System Performance (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console. As for your general organization quotas and usage, they appear on the License & Usage (platform-ca | platform-eu | platform-au). See Monitor system performance and Review your license and usage for details on these pages.
To upgrade your license or subscribe to another Coveo product, contact your Coveo representative.
Request size constraints
You will typically get a 413 Request Entity Too Large response if your total Push API request size exceeds:
-
256 MB when pushing a large file container. See Manage batches of items in a Push source and Manage batches of security identities.
-
6 MB when pushing a single item or security identity. See Add or update a single item in a Push source, Add or update a single security identity, and Add or update a single alias.
Item entitlement
You will receive a 412 DOCUMENT_LIMIT_EXCEEDED response when your index item entitlement has been reached.
You can review the number of items in your index, as well as your entitlement, on the System Performance (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console.
See Troubleshooting Push source issues for more information.
Item permission requirements
When the Same users and groups as in your content system content security option is used, the Push API enforces strict requirements regarding the number of permissions and the validity of security identities in any request to add or update items:
-
Each item permission must reference a valid security identity.
-
An item can contain a maximum of 5,000 permissions.
If these requirements are not met, the Push API will return a 202 Accepted response, but the item will eventually be rejected.
Check the Log Browser documentation to learn how to determine whether an item has been properly processed or not.
Push API daily quotas
The Push API returns a 429 response:
-
TOO_MANY_REQUESTSif you exceed your organization’s daily request quota. -
TOO_MANY_DOCUMENTS_DAILYif you exceed your daily item processing quota.
See Troubleshooting Push source issues for more information on these errors.
You can review your Coveo usage and quotas on the System Performance (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console. This page also displays several metrics related to your Push API usage.
Maximum number of items per time period
Coveo enforces maximum numbers of items that you can push over specific time periods to ensure you don’t use up your daily quotas in a very short time and to maintain optimal service performance for all customers.
Assuming you’re using a fairly small number of batch calls to do so, you can push no more than the following number of items over the specified time periods:
-
600,000 items over a five-minute period burst. If you exceed this number, you’ll receive a
429 DOCUMENT_UPDATE_RATE_LIMIT_EXCEEDEDresponse. [1] This quota appears on the License & Usage (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console, along with your usage. -
1,000,000 items over a one-hour period burst.
-
300,000 items over a one-hour sustained flow.
|
|
Always use batch requests to push multiple items or security identities. Remember that contractually, you must contact us to arrange performance tests on Coveo. |
The index speed of a Coveo organization varies based on its typical usage. Also, the more items an index contains, the longer it takes to process new items. A typical production index can process up to 300,000 items per hour, while a typical trial index can process no more than 10,000 items per hour.
While you technically can push bursts of up to 1,000,000 items over a one-hour period, your index will typically require at least 3 hours to process all this new information. Similarly, while you can push a sustained flow of 300,000 items per hour, there’s no guarantee that your index will be able to keep up with that pace.
When pushing many items over a short period of time, buffering typically occurs.
Security identity requirements
The Push API enforces strict requirements regarding the number and validity of security identities and their relationships in any request to add or update security identities:
-
Every security identity, including those referenced in relationship definitions, must be valid.
-
A single security identity can’t have more than 100,000 relationships.
If these requirements aren’t met, the API returns a 202 Accepted response, but the payload isn’t processed.
Security identity daily quotas
You’ll receive a 429 TOO_MANY_REQUESTS response if you exceed your organization’s daily quota on Push API requests.
There’s also a daily quota on the number of identity relationships the Push API can process. The total number of relationships associated with an identity is defined as the sum of all members, aliases, and granted identities appearing in the body of a request to add or update security identities or security identity aliases. Specifically, the total number of relationships is computed as follows:
-
When the request body implements the
IdentityBodymodel, the total number of relationships is the sum of its number ofmembersandwellKnowns. -
When the request body implements the
MappedIdentityBodymodel, the total number of relationships is the sum of its number ofmappingsandwellKnowns.
Every time you push an identity, even if it remains the same, the number of relationships it defines contributes to the daily count.
You can review the number of identity relationships processed, as well as your daily quota, on the License & Usage (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console.
You’ll receive a 429 TOO_MANY_RELATIONSHIPS_DAILY response if you exceed this quota.
|
|
You shouldn’t be able to reach this quota in a properly configured Coveo implementation. If you do manage to reach it, you won’t be able to push any more relationships until your daily count resets, which may cause permission mismatches on items. |
To ensure you don’t use up your daily quota in a very short time and to maintain optimal service performance for all customers, Coveo also enforces a quota of security identity relationship updates over a five-minute period.
You can review your usage of this quota on the License & Usage (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console.
If you exceed this quota, you’ll receive a 429 RELATIONSHIP_UPDATE_RATE_LIMIT_EXCEEDED response.[2]
Example: Computing the number of relationships of an identity
Consider the following request body:
[
{
"identity": {
"name": "Group_A",
"type": "GROUP",
},
"members": [
{
"name": "Group_B",
"type": "GROUP",
}
],
"wellKnowns": [
{
"name": "Company",
"type": "GROUP",
}
]
},
{
"identity": {
"name": "Group_B",
"type": "GROUP",
},
"members": [
{
"name": "user1@example.com",
"type": "User",
},
{
"name": "user2@example.com",
"type": "User",
}
],
"wellKnowns": []
}
]
The definition of Group_A in the above request includes one member and one wellknown.
Group_A therefore includes two security identity relationships.
The definition of Group_B includes two members and no wellknowns.
Group_B therefore includes two security identity relationships.
Therefore, the above request pushes four security identity relationships in total.
Security identity validation
The Push API validates and rejects invalid security identities. For an identity to be valid, it must meet the following criteria:
-
Identity
namevalue:-
Must be between 1 and 250 characters long
-
Must not start or end with a blank character
-
Must not contain control characters (for example, tabs or new line characters)
-
-
Identity
additionalInfokeys:-
Must be at least 1 character long
-
Must not start or end with a blank character
-
Must not contain control characters (for example, tabs or new line characters)
-
-
Total identity size: The total size of a security identity is defined as the sum of the characters in its
namevalue and the combined number of characters used by all keys and values in theadditionalInfosection. The total identity size must not exceed 500 characters.
Example: Computing the total identity size
Consider the following request body:
[
{
"identity": {
"name": "User1",
"type": "USER",
"additionalInfo": {
}
},
"members": [],
"wellKnowns": []
},
{
"identity": {
"name": "Group1",
"type": "GROUP",
"additionalInfo": {
"key": "value",
"another_key": "another_value"
}
},
"members": [],
"wellKnowns": []
}
]
The total size of the User1 identity is five, because its name value contains five characters and its additionalInfo section is empty.
The total size of the Group1 identity is 38, because its name value contains six characters and its additionalInfo section contains 32 characters (that is, 3+5+11+13).