--- title: Manage batches of items in a Push source slug: '90' canonical_url: https://docs.coveo.com/en/90/ collection: index-content source_format: adoc --- # Manage batches of items in a Push source :figure-caption!: Managing [items](https://docs.coveo.com/en/210/) in batches is the best way to reduce the risk of getting throttled by the service. For details about the Push API rate limits, see [Maximum number of items per time period](https://docs.coveo.com/en/63#maximum-number-of-items-per-time-period). _Batch_ Push API requests allow you to forward many item updates to the service using only a few Push API requests, rather than performing hundreds (or thousands) of _single_ item requests to achieve the same results. Performing batch update Push API requests is slightly more complex than performing single Push API requests, as doing so involves three distinct steps which are detailed in this article. > **Leading practice: Perform a security identity update** > > Whenever you add or update content in a secured Push [source](https://docs.coveo.com/en/246/), you should ensure that the [security identities](https://docs.coveo.com/en/240/) referenced in item [permission models](https://docs.coveo.com/en/225/) are up to date in the [security identity cache](https://docs.coveo.com/en/241/). > > In essence, you should always perform a [security identity update](https://docs.coveo.com/en/244/) on the [security identity provider](https://docs.coveo.com/en/242/) of a secured Push source _before_ you perform a content update in that source. > For more information, see [Manage security identities in a security identity provider](https://docs.coveo.com/en/132/). ## Step 1: Create a file container Perform a [`Create a file container`](https://docs.coveo.com/en/43/) HTTP request. ## Step 2: Upload the content update into the file container Perform the following `PUT uploadUri` request to upload the content update into the Amazon S3 file container you got from [step 1](#step-1-create-a-file-container). **Sequence diagram highlighting the `PUT uploadUri` request in the context of a batch update sequence and illustrating how the [Coveo indexing pipeline](https://docs.coveo.com/en/1893/) handles the batch update.** ![Sequence diagram for PUT upload URI request](https://docs.coveo.com/en/assets/images/index-content/put-uploaduri-sequence-diagram.svg) ### Request template ```http PUT HTTP/1.1 ​ ``` Request parameters: **Parameters**
Details * Replace `` with the value of the `uploadUri` property you got in the response when you created your file container in [step 1](#step-1-create-a-file-container). * Replace `` with the key-value pairs of the `requiredHeaders` object property you got in the response when you created your file container in [step 1](#step-1-create-a-file-container).
Request body: ```json { "addOrUpdate": [ { *, "documentId": , <"data"|"compressedBinaryData"|"compressedBinaryDataFileId">: , "compressionType": <"UNCOMPRESSED"|"DEFLATE"|"GZIP"|"LZMA"|"ZLIB">, "fileExtension": , "parentId": , "permissions": }* ], "delete": [ { "documentId": , "deleteChildren": }* ] } ``` **Parameters**
Details The request body must implement the [`BatchDocumentBody` model](https://docs.coveo.com/en/78#batchdocumentbody-model). For each item you include in the `addOrUpdate` array: * Replace `*` with any number of arbitrary [metadata key-values](https://docs.coveo.com/en/115/) you want to include along with the item you're adding or updating. * Replace `` with the URI of the item to add or update (for example, `+http://www.example.com/toadd.html+`). Specifying a unique `documentId` value for each item is mandatory. * Replace `<"data"|"compressedBinaryData"|"compressedBinaryDataFileId">` with the property you want to use to push the item data. You must also replace `` accordingly. For more information, see [Pushing item data](https://docs.coveo.com/en/73/). * If you're using the [`compressedBinaryData`](https://docs.coveo.com/en/164/) or the [`compressedBinaryDataFileId`](https://docs.coveo.com/en/69/) property to push item data, replace `<"UNCOMPRESSED"|"DEFLATE"|"GZIP"|"LZMA"|"ZLIB">` with the actual compression algorithm that was applied to the item data. Otherwise, you don't need to include the `compressionType` property. * (Recommended) Replace `` with the actual file extension which the Push API should use to interpret the item data (for example, `.txt`, `.html`, etc.). This value must include the leading dot (`.`) character. * To [establish a parent-child relationship](https://docs.coveo.com/en/57#define-parent-child-relationships) between the item and other items in the same Push source, replace `` with the `documentId` of the parent item, or the `documentId` of the item you're adding or updating if this item is the parent. Otherwise, you don't need to include the `parentId` property in your request body. * If the target Push source is secured, replace `` with a valid permission model for the pushed item. See [Push API reference - Item permission models](https://docs.coveo.com/en/78#item-permission-models), [Simplified permission model](https://docs.coveo.com/en/107/), and l[Complete permission model](https://docs.coveo.com/en/25/) for details. Otherwise, you don't need to include the `permissions` property. For each item you include in the `delete` array: * Replace `` with the URI of the item to delete (for example, `+http://www.example.com/todelete.html+`). * Set the `deleteChildren` property to `true` to delete all items whose `documentId` is nested under the `documentId` to delete. The `deleteChildren` property is set to `false` by default.
Successful response: `200 OK` A successful response has no content, but indicates that the content update was successfully uploaded to the Amazon S3 file container. ### Sample request The following example shows how to upload a batch of item updates into a file container. ```http PUT link:https://coveo-nprod-customerdata.s3.amazonaws.com/proda/blobstore/mycoveocloudv2organizationg8tp8wu3/b5e8767e-8f0d-4a89-9095-1127915c89c7[...] HTTP/1.1 ​ x-amz-server-side-encryption: AES256 Content-Type: application/octet-stream ``` Request body: ```json { "addOrUpdate": [ { // ...Metadata... "documentId": "http://www.example.com/toadd.html", "compressedBinaryData": "ZUp5enlTakp6YkhqVWdBQ200elV4QlFvVXgrSm5aU2ZVZ2xoUWxRWjJybFdKT1lXNUtRQ1ZSa2lTUlRZaFdSa0Zpc0FVV0tlUWlwRWlaNk5mb0VkRjZlTlBzUVFvQTZRZFFETWhCOEU=", "compressionType": "DEFLATE", "fileExtension": ".html", "parentId": "http://www.example.com/", "permissions": [ { "allowAnonymous": false, "allowedPermissions": [ // ...Allowed security identities... ], "deniedPermissions": [ // ...Denied security identities... ] }, // ...More permission sets... ] }, // ...More items to add or update... ], "delete": [ { "documentId": "http://www.example.com/todelete.html", "deleteChildren": true }, // ...More items to delete... ] } ``` ## Step 3: Push the file container into a Push source Use the `Add, update, and/or delete a batch of items` request to push the Amazon S3 file container into a Push source. **Sequence diagram highlighting the `Add, update, and/or delete a batch of items` request in the context of a batch update sequence and illustrating how the [Coveo indexing pipeline](https://docs.coveo.com/en/1893/) handles the batch update.** ![Sequence diagram for PUT batch request](https://docs.coveo.com/en/assets/images/index-content/put-batch-sequence-diagram.svg) ### Request template **US East region** [%collapsible%open] #### [source,http,subs=attributes] ``` PUT https://api.cloud.coveo.com/push/v1/organizations//sources//documents/batch?fileId= HTTP/1.1 ​ Accept: application/json Authorization: Bearer ``` #### .Canada region
Details [source,http,subs=attributes] ``` PUT https://api-ca.cloud.coveo.com/push/v1/organizations//sources//documents/batch?fileId= HTTP/1.1 ​ Accept: application/json Authorization: Bearer ```
**Ireland region**
Details [source,http,subs=attributes] ``` PUT https://api-eu.cloud.coveo.com/push/v1/organizations//sources//documents/batch?fileId= HTTP/1.1 ​ Accept: application/json Authorization: Bearer ```
**Australia region**
Details [source,http,subs=attributes] ``` PUT https://api-au.cloud.coveo.com/push/v1/organizations//sources//documents/batch?fileId= HTTP/1.1 ​ Accept: application/json Authorization: Bearer ```
Request parameters: **Parameters**
Details In the request path: * Replace `` with your [organization ID](https://docs.coveo.com/en/148/). * Replace `` with the [ID](https://docs.coveo.com/en/3390#copy-a-source-name-or-id) of the target Push source. In the query string: * Replace `` with the `filedId` you got from [step 1](#step-1-create-a-file-container). In the `Authorization` HTTP header: * Replace `` with a Push source API key that grants the [set of privileges required to push items](https://docs.coveo.com/en/1546#api-key) to your source.
Request body: None Successful response: `202 Accepted` A successful response (`202 Accepted`) has no content, but indicates that the operation was successfully forwarded to the service and that the batch of items is now enqueued to be processed by the [Coveo indexing pipeline](https://docs.coveo.com/en/184/). This doesn't imply that all items in the batch were successfully added, updated and/or deleted in the target Push source. See [About the Push API processing delay](https://docs.coveo.com/en/32/) for details. > **Tip** > > The contents of a file container can be pushed to multiple sources or security identity providers in the same [Coveo organization](https://docs.coveo.com/en/185/). > Just update the target `sourceId`/`providerId` and Authorization HTTP header access token in your other item or [security identity batch requests](https://docs.coveo.com/en/55#step-3-push-the-file-container-into-a-security-identity-provider). > > The file container remains available for 4 days. ### Sample request The following example shows how to push a file container into a Push source. ```http PUT https://api.cloud.coveo.com/push/v1/organizations/mycoveocloudv2organizationg8tp8wu3/sources/mycoveocloudv2organization-rp5rxzbdz753uhndklv2ztkfgy/documents/batch?fileId=b5e8767e-8f0d-4a89-9095-1127915c89c7 HTTP/1.1 ​ Authorization: Bearer **********-****-****-****-************ Accept: application/json ```