---
title: Partial catalog data updates
slug: p4eb0515
canonical_url: https://docs.coveo.com/en/p4eb0515/
collection: coveo-for-commerce
source_format: adoc
---
# Partial catalog data updates

To partially update your [catalog data](https://docs.coveo.com/en/obcf0333/) in your source (typically a [Catalog source](https://docs.coveo.com/en/l5if0244/)), you have to interact with the [Coveo Stream API](https://docs.coveo.com/en/12#tag/Stream).
The [Coveo Platform](https://docs.coveo.com/en/186/) supports two types of operations to perform partial updates on your [catalog data](https://docs.coveo.com/en/obcf0333/):

* [Partial item updates](#partial-item-updates) let you update one or more [fields](https://docs.coveo.com/en/200/) in an existing source.
Using this method reduces the number of calls made to the [index](https://docs.coveo.com/en/204/) and removes the need to [rebuild](https://docs.coveo.com/en/2712/) the entire product list.

For example, you may want to use the _partial item update_ mechanism to:

** Update a price field across a selection of documents.

** Add or remove a [variant](https://docs.coveo.com/en/mc7f0326/) from a store array.

** Update the value of an existing product field.

** Update a dictionary field.

* [Shallow merge operations](#shallow-merge-operations): This operation allows you to update one or more [fields](https://docs.coveo.com/en/200/) in a [catalog object](https://docs.coveo.com/en/ncig0154/).
If the item doesn't exist, it will be created, and if the [field](https://docs.coveo.com/en/200/) is missing, it will be added.
In other words, shallow merge operations:

** Add [metadata](https://docs.coveo.com/en/218/) to existing [items](https://docs.coveo.com/en/pa8f6515/).

** Update existing [metadata](https://docs.coveo.com/en/218/) on [items](https://docs.coveo.com/en/pa8f6515/).

** Preserve existing [metadata](https://docs.coveo.com/en/218/) values that aren't specified in the payload of the merge operation.

** [index](https://docs.coveo.com/en/204/) new [items](https://docs.coveo.com/en/pa8f6515/) if the [product identifier](https://docs.coveo.com/en/n73f0502/) mentioned in the payload doesn't already exist in the [source](https://docs.coveo.com/en/246/).
Unlike [partial item updates](https://docs.coveo.com/en/p4eb0515#partial-item-updates), this method allows you to index items for the first time.
However, the [full item update operation](https://docs.coveo.com/en/p4eb0129/) is still the recommended approach if you're only indexing new items.

## Leading practices

Any partial update to your [catalog data](https://docs.coveo.com/en/obcf0333/) should be performed using the [partial item update](#partial-item-updates) operation.
This operation is the most efficient way to update your [catalog data](https://docs.coveo.com/en/obcf0333/) in your source.

[Shallow merge operations](#shallow-merge-operations) should be used only when it's impossible to validate the existence of an [item](https://docs.coveo.com/en/pa8f6515/) in the Catalog source before partially updating it.

## Prerequisites

To perform the operations listed in this article, you must have:

* Created a [Catalog source](https://docs.coveo.com/en/n8of0593/) or [equivalent stream-enabled source](https://docs.coveo.com/en/3448#alternatives-to-the-catalog-source).
Shallow merge operations are only supported on Catalog sources.

* Created and configured a [catalog entity](https://docs.coveo.com/en/3139/) that uses the source containing your [catalog data](https://docs.coveo.com/en/obcf0333/).

## Partial item updates

Partial item updates let you update one or more [fields](https://docs.coveo.com/en/200/) in an existing source.
Using this method reduces the number of calls made to the [index](https://docs.coveo.com/en/204/) and removes the need to [rebuild](https://docs.coveo.com/en/2712/) the entire product list.

For example, you may want to use the partial item update mechanism to:

** [Update a price field across a selection of documents](#replace-a-field-value).

** [Add](#add-items) or [remove](#remove-items) a [variant](https://docs.coveo.com/en/mc7f0326/) from a store array.

** [Update the value of an existing product field](#replace-a-field-value).

** [Update a dictionary field](#dictionary-field-update).

To perform a partial [item](https://docs.coveo.com/en/pa8f6515/) update, you must interact with the [Coveo Stream API](https://docs.coveo.com/en/12#tag/Stream).
This section guides you through the different actions that must be taken to update your [items](https://docs.coveo.com/en/pa8f6515/).

### Structure and operations

The following example represents the basic structure of a JSON file to send for a partial [item](https://docs.coveo.com/en/pa8f6515/) update:

```json
{
  "partialUpdate": [
    {
      "documentId": "<DOCUMENT_ID>", <1>
      "operator": "<PARTIAL_UPDATE_OPERATOR>", <2>
      "field": "<FIELD_NAME>", <3>
      "value": <VALUE> <4>
    },
  ]
}
```

Where you replace:

<1> `<DOCUMENT_ID>` with the required unique identifier of the document to apply a partial update operation to.
Therefore, you should make sure that all of your [items](https://docs.coveo.com/en/pa8f6515/) contain a `documentId` for which the value is a URI that uniquely identifies the [item](https://docs.coveo.com/en/pa8f6515/).
This value must be a valid URL with a proper URI prefix, such as `product://`, or any other scheme that fits your [catalog data](https://docs.coveo.com/en/obcf0333/).

<2> `<PARTIAL_UPDATE_OPERATOR>` with one of the following operators.
Note that they're treated sequentially:

[%header,cols="1,2"]
|===
|Operator
|Description

|`arrayAppend`
|Adds a list of elements to an array attribute.
For example, adding unique identifiers for products or variants to a store, or making products or variants available.

|`arrayRemove`
|Removes a list of elements from an array attribute.
For example, removing unique identifiers for products or variants from a store, or making products or variants unavailable.

|`fieldValueReplace`
|Replaces a [field](https://docs.coveo.com/en/200/) value regardless of the original [value type](https://docs.coveo.com/en/1833#field-type).

|`dictionaryPut`
a|Adds a key-value pair to a given dictionary [field](https://docs.coveo.com/en/200/).

If the key already exists in the dictionary, its value will be updated.
If the dictionary doesn't exist yet on the item, it will be automatically added.

```json
{
  "key1": "value1"
}
```

|`dictionaryRemove`
a|Removes a key-value pair as long as the key value already exists.

**Removing a single key:**

```json
"value": "key1"
```

**Removing multiple keys:**

```json
"value": [ "key1", "key2", "Key3" ]
```
|===


> **WARNING**
>
> If an operation in the partial [item](https://docs.coveo.com/en/pa8f6515/) update is found to be invalid, it will be ignored.

<3> `<FIELD_NAME>` with the [field](https://docs.coveo.com/en/200/) name you wish to update. The `<FIELD_NAME>` must be at most 1024 characters long.

<4> `<VALUE>` depending on the chosen `<PARTIAL_UPDATE_OPERATOR>`, the `<VALUE>` can differ:

** To add [items](https://docs.coveo.com/en/pa8f6515/), see [Add items](#add-items)

** To remove [items](https://docs.coveo.com/en/pa8f6515/), see [Remove items](#remove-items)

** To replace a [field](https://docs.coveo.com/en/200/) value, see [Replace a field value](#replace-a-field-value)

** To update a dictionary [field](https://docs.coveo.com/en/200/), see [Dictionary field update](#dictionary-field-update)

#### About empty values

If you omit the `value` field in a partial update operation, the system treats it as if `null` was sent.
This results in the field being removed from the item.

For example, these two operations are equivalent:

```json
{
  "documentId": "sku://mt001",
  "operator": "fieldValueReplace",
  "field": "partialTag"
}
```

```json
{
  "documentId": "sku://mt001",
  "operator": "fieldValueReplace",
  "field": "partialTag",
  "value": null
}
```

Both operations will remove the `partialTag` field from the item.

#### Protected fields

The following [fields](https://docs.coveo.com/en/200/) aren't updated during a partial update operation:

* `documentid`

* `permanentid`

* Product, [variant](https://docs.coveo.com/en/mc7f0326/), or [availability](https://docs.coveo.com/en/mc7e9096/) object type [fields](https://docs.coveo.com/en/200/)

* Product, [variant](https://docs.coveo.com/en/mc7f0326/), or [availability](https://docs.coveo.com/en/mc7e9096/) ID [fields](https://docs.coveo.com/en/200/)

Object types and ID [fields](https://docs.coveo.com/en/200/) may vary depending on your [catalog configuration](https://docs.coveo.com/en/3139#catalog-configuration).

### Step 1: Create a file container (partial item update)

> **Important**
>
> Make sure that you meet the [prerequisites](#prerequisites) before performing this operation.

To perform a merge document update, you must first [create an Amazon S3 file container](https://docs.coveo.com/en/43/).
Use the [Create a file container](https://docs.coveo.com/en/12/api-reference/push-api#tag/File-Container/paths/~1organizations~1%7BorganizationId%7D~1files/post) operation to create an Amazon S3 file container for a specific [Coveo organization](https://docs.coveo.com/en/185/):

**Request template**

```http
POST https://api.cloud.coveo.com/push/v1/organizations/<MyOrganizationId>/files?useVirtualHostedStyleUrl=<true|false> HTTP/1.1
​
Accept: application/json
Content-Type: application/json
Authorization: Bearer <MyAccessToken>
```

In the request path:

* Replace `<MyOrganizationId>` with your [organization ID](https://docs.coveo.com/en/148/).

In the query string:

* Optionally, set `useVirtualHostedStyleUrl` to `true` if you want the service to return a virtual hosted-style URL, such as `+coveo-nprod-customerdata.s3.amazonaws.com/...+`.
The default value is currently `false`, which means that the service returns path-style URLs, such as `+s3.amazonaws.com/coveo-nprod-customerdata/...+`.

> **Important**
>
> The `useVirtualHostedStyleUrl` query string parameter will soon be deprecated as part of the [path-style URL deprecation](https://docs.coveo.com/en/43#path-style-url-deprecation).
> From this point onwards, the service will only return virtual hosted-style URLs.

In the `Authorization` HTTP header:

* Replace `<MyAccessToken>` with an access token, such as an [API key](https://docs.coveo.com/en/1718/) that has the [required privileges](#required-privileges) to push content to the source.

**Payload**: None

The body of a successful response contains important information about the temporary, private, and encrypted Amazon S3 file container that you just created:

```json
{
    "uploadUri": "<UPLOAD_URI>", <1>
    "fileId": "<FILE_ID>", <2>
    "requiredHeaders": { <3>
        "x-amz-server-side-encryption": "AES256",
        "Content-Type": "application/octet-stream"
    }
}
```

<1> The `uploadUri` property contains a pre-signed URI to
use in the PUT request of [step 2](#step-2-upload-the-merge-item-content-into-the-file-container).

> **Notes**
>
> * The Amazon S3 file container applies [AES-256](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) server-side encryption to your data.
> 
> * The file container is automatically deleted as soon as its content has been successfully forwarded to the service.
> 
> * The `uploadUri` automatically expires after 60 minutes.
> 
> Therefore, it's safe to upload sensitive information into the Amazon S3 file container.

<2> The `fileId` property contains the unique identifier of your file container.
You must use this value to send the file container to the source in step 3.
<3> The `requiredHeaders` property contains the required HTTP headers for sending
in the PUT request of [step 2](#step-2-upload-the-merge-item-content-into-the-file-container).

### Step 2: Upload the partial item content into the file container

To upload the update file into the Amazon S3 file container you got from [step 1](#step-1-create-a-file-container-partial-item-update), perform the following PUT request:

**Request template**

```http
PUT <MyUploadURI> HTTP/1.1
​
<HTTPHeaders>
```

Where you replace:

* `<MyUploadURI>` with the value of the `uploadUri` property you got in the response when you created your file container in [step 1](#step-1-create-a-file-container-partial-item-update).

* `<HTTPHeaders>` with the key-value pairs of the `requiredHeaders` object property you got in the response when you created your file container in [step 1](#step-1-create-a-file-container-partial-item-update).

You can now upload your update data in the body of the request.
You must structure the data in JSON format and upload it as either a JSON file or a file compressed with the ZLib algorithm.
Expand the following sections to see examples of how to structure your JSON file for each of the available operations:

**Add items**
<details><summary>Details</summary>

[discrete]

</details> Add items
The following example shows how to make products or variants available for a store:

```json
{
  "partialUpdate": [
    {
      "documentId": "store://s000001",
      "operator": "arrayAppend",
      "field": "ec_available_items",
      "value": ["sku-224", "sku-225"]
    },
  ]
}
```
#### .Remove items
<details><summary>Details</summary>

[discrete]

</details> Remove items
The following example shows how to remove products or variants from a store:

```json
{
  "partialUpdate": [
    {
      "documentId": "store://s000001",
      "operator": "arrayRemove",
      "field": "ec_available_items",
      "value": ["sku-221", "sku-220"]
    },
  ]
}
```
#### .Replace a field value
<details><summary>Details</summary>

[discrete]

</details> Replace a field value

> **Note**
>
> When performing an update to the value of a product [field](https://docs.coveo.com/en/200/) that doesn't exist, the [field](https://docs.coveo.com/en/200/) will be created.

The following example shows how to modify a price and a description of a specific [item](https://docs.coveo.com/en/pa8f6515/), as well as change the name of another [item](https://docs.coveo.com/en/pa8f6515/) during the same update:

```json
{
  "partialUpdate": [
    {
      "documentId": "product://010",
      "operator": "fieldValueReplace",
      "field": "ec_price",
      "value": 23.50
    },
    {
      "documentId": "product://010",
      "operator": "fieldValueReplace",
      "field": "ec_description",
      "value": "Written representation of the change in description"
    },
    {
      "documentId": "product://040",
      "operator": "fieldValueReplace",
      "field": "ec_name",
      "value": "Product name X"
    },
  ]
}
```
#### .Dictionary field update
<details><summary>Details</summary>

[discrete]

</details> Dictionary field update

You can use the [partial item updates](#partial-item-updates) method to partially update [dictionary fields](https://docs.coveo.com/en/2036/) on a [Catalog source](https://docs.coveo.com/en/l5if0244/) [item](https://docs.coveo.com/en/pa8f6515/).
When performing this update, if the dictionary [field](https://docs.coveo.com/en/200/) to update doesn't exist, the [field](https://docs.coveo.com/en/200/) will be added as a single-[item](https://docs.coveo.com/en/pa8f6515/) dictionary.
The following examples show how to update dictionary [fields](https://docs.coveo.com/en/200/) for [items](https://docs.coveo.com/en/pa8f6515/) of the product [catalog object](https://docs.coveo.com/en/ncig0154/).
However, this method is applicable to any other [catalog object](https://docs.coveo.com/en/ncig0154/) type, such as variants or availability.

> **Important**
>
> You should exercise caution when using distinct HTTP calls to send multiple updates targeting the same dictionary [field](https://docs.coveo.com/en/200/) in rapid succession.
> If an update to apply is received, and is older than the most recently applied update, the system will ignore the older message.
> This emphasizes the importance of ensuring proper order and sequencing when sending updates to prevent unintentional data inconsistencies.

Consider this initial product [metadata](https://docs.coveo.com/en/218/).
It contains dictionary [field](https://docs.coveo.com/en/200/) key-value pairs of types of ice-cream and their respective unique identifiers:

```json
{
  "addOrUpdate": [
    {
      "documentId": "product://ice-cream_0001",
      "objecttype": "Product",
      "ec_product_id": "ice-cream_0001",
      "ec_price": 9.99,
      "flavors": {
        "Vanilla": "SKU-0001",
        "Chocolate": "SKU-0002",
        "Pumpkin": "SKU-0003",
        "Cinnamon": "SKU-0004",
        "Candies": "SKU-0005"
      }
    }
  ]
}
```

The user decides to add a new flavor, `"Moka": "SKU-123456"`, to the existing list.
The JSON file of the partial dictionary update would look like the following:

```json
{
  "partialUpdate": [
    {
      "documentId": "product://ice-cream_0001",
      "operator": "dictionaryPut",
      "field": "flavors",
      "value": {
        "Moka": "SKU-123456"
      }
    }
  ]
}
```

After which, the resulting updated dictionary would look like this:

```json
{
  "documentId": "product://ice-cream_0001",
  "objecttype": "Product",
  "ec_product_id": "ice-cream_0001",
  "ec_price": 9.99,
  "flavors": {
    "Vanilla": "SKU-0001",
    "Chocolate": "SKU-0002",
    "Pumpkin": "SKU-0003",
    "Cinnamon": "SKU-0004",
    "Candies": "SKU-0005",
    "Moka": "SKU-123456"
  }
}
```

To remove a key, such as the `"Pumpkin"` flavor, from the existing list, you need only identify the key as follows:

```json
{
  "partialUpdate": [
    {
      "documentId": "product://ice-cream_0001",
      "operator": "dictionaryRemove",
      "field": "flavors",
      "value": "Pumpkin"
    }
  ]
}
```

To remove multiple keys, such as the `"Pumpkin"`, `"Cinnamon"`, and `"Candies"` flavors, from the dictionary, enter all of the keys in an array as follows:

```json
{
  "partialUpdate": [
    {
      "documentId": "product://ice-cream_0001",
      "operator": "dictionaryRemove",
      "field": "flavors",
      "value": [ "Pumpkin", "Cinnamon", "Candies" ]
    }
  ]
}
```
#### A successful response has no content, but indicates that the content update was successfully uploaded to the Amazon S3 file container, as follows:

**200 OK**

```json
{}
```

> **Important**
>
> When the payload exceeds 256 MB, it must be chunked into 256 MB parts.
> See [Uploading large catalog data files](#uploading-large-catalog-data-files) for instructions.

### Step 3: Send the file container to update your source (partial item update)

To push the Amazon S3 file container into your [source](https://docs.coveo.com/en/246/), use the [Update a catalog stream source](https://platform.cloud.coveo.com/docs?urls.primaryName=PushAPI#/Stream/put_organizations%5F%5ForganizationId%5F%5Fsources%5F%5FsourceId%5F%5Fstream_update) operation as follows:

**Request template**

```http
PUT https://api.cloud.coveo.com/push/v1/organizations/<MyOrganizationId>/sources/<MySourceId>/stream/update?fileId=<MyFileId> HTTP/1.1
​
Content-Type: application/json
Authorization: Bearer <MyAccessToken>
```

**Payload**

```json
{}
```

Where you replace:

* `<MyOrganizationId>` with the ID of the target [Coveo organization](https://docs.coveo.com/en/185/) (see [Retrieve the organization ID](https://docs.coveo.com/en/148/)).

* `<MySourceId>` with the [ID](https://docs.coveo.com/en/3390#copy-a-source-name-or-id) of the [source](https://docs.coveo.com/en/246/) which contains the [catalog data](https://docs.coveo.com/en/obcf0333/) that you want to update.

* `<MyFileId>` with the `fileId` you got from step 1.

* `<MyAccessToken>` with an access token, such as an [API key](https://docs.coveo.com/en/1718/) that has the [required privileges](#required-privileges) to push content to the source.

A successful response (`202`) indicates that the operation was successfully forwarded to the service and that the batch of [items](https://docs.coveo.com/en/pa8f6515/) is now enqueued to be processed by the [Coveo indexing pipeline](https://docs.coveo.com/en/184/).
For example:

**202 Accepted**

```text
{
  "orderingId": 1716387965000, <1>
  "requestId": "498ef728-1dc2-4b01-be5f-e8f8f1154a99" <2>
}
```

Where:

<1> [`orderingId`](https://docs.coveo.com/en/147/) indicates the time your request was received.

<2> `requestId` is the unique identifier for your request.

> **Tip**
>
> The contents of a file container can be pushed to multiple sources in the same [Coveo organization](https://docs.coveo.com/en/185/).
> Just update the target `sourceId` and Authorization HTTP header access token in your other Stream API update or [merge requests](https://docs.coveo.com/en/p4eb0515#shallow-merge-operations).
> 
> The file container remains available for 4 days.

### Example

The following is a complete example of a _partial_ update used to update documents in a [source](https://docs.coveo.com/en/246/).

```json
{
  "partialUpdate": [
    {
      "documentId": "store://s000003",
      "operator": "arrayAppend",
      "field": "ec_available_items",
      "value": [
        "sku-224",
        "sku-225"
      ]
    },
    {
      "documentId": "store://s000005",
      "operator": "arrayRemove",
      "field": "ec_available_items",
      "value": [
        "sku-221",
        "sku-220"
      ]
    },
    {
      "documentId": "product://030",
      "operator": "fieldValueReplace",
      "field": "ec_price",
      "value": 23.5
    }
  ]
}
```

## Shallow merge operations

The [Catalog source](https://docs.coveo.com/en/l5if0244/) supports _shallow merge operations_, enabling updates to one or more [fields](https://docs.coveo.com/en/200/) in a [catalog object](https://docs.coveo.com/en/ncig0154/).
If the item doesn't exist, it will be created, and if the field is missing, it will be added.

This operation should be used when it's impossible to validate the existence of an [item](https://docs.coveo.com/en/pa8f6515/) in the Catalog source before partially updating it.

The _merge_ aspect means that the operation allows for updating one or multiple fields on an item that already exists in the Catalog source.
If the item specified in the payload of the operation doesn't exist in the source, the item is created.

The _shallow_ aspect means that the operation doesn't allow for partial updates on dictionary fields as it overwrites the entire value(s) of the field.
See [Shallow merge operations and dictionary fields](#shallow-merge-operations-and-dictionary-fields) for more information about the effect of using shallow merge operations with dictionary fields.

In other words, shallow merge operations:

* Add [metadata](https://docs.coveo.com/en/218/) to existing [items](https://docs.coveo.com/en/pa8f6515/).

* Update existing [metadata](https://docs.coveo.com/en/218/) on [items](https://docs.coveo.com/en/pa8f6515/).

* Preserve existing [metadata](https://docs.coveo.com/en/218/) values that aren't specified in the payload of the merge operation.

* [index](https://docs.coveo.com/en/204/) new [items](https://docs.coveo.com/en/pa8f6515/) if the [product identifier](https://docs.coveo.com/en/n73f0502/) mentioned in the payload doesn't already exist in the [source](https://docs.coveo.com/en/246/).
Unlike [partial item updates](https://docs.coveo.com/en/p4eb0515#partial-item-updates), this method allows you to index items for the first time.
However, the [full item update operation](https://docs.coveo.com/en/p4eb0129/) is still the recommended approach if you're only indexing new items.

> **Warning**
>
> Shallow merge operations shouldn't be used to append or remove values from a [dictionary field](https://docs.coveo.com/en/2036/).
> See [Shallow merge operations and dictionary fields](#shallow-merge-operations-and-dictionary-fields) for more information about the effect of using shallow merge operations with dictionary fields.

### Usage example

Consider an item of the `Product` [catalog object](https://docs.coveo.com/en/ncig0154/), configured with the following [metadata](https://docs.coveo.com/en/218/):

```json
{
  "objecttype": "Product",
  "documentId": "product://010",
  "ec_name": "Sneaker 010",
  "productId": "010",
  "ec_category": "Sneakers",
  "gender": "Unisex",
  "department": "Shoes"
}
```

You can perform a shallow merge operation to add a new field to this item, such as `ec_brand`, and update the `gender` field.

```json
{
  "addOrMerge": [
    {
      "documentId": "product://010",
      "gender": "Women",
      "ec_brand": "ACME"
    }
  ]
}
```

In this example, the `ec_brand` field is added to the item, and the `gender` field is updated to `Women`.
The item now appears in the [source](https://docs.coveo.com/en/246/) with the following [metadata](https://docs.coveo.com/en/218/):

```json
{
  "objecttype": "Product",
  "documentId": "product://010",
  "ec_name": "Sneaker 010",
  "productId": "010",
  "ec_category": "Sneakers",
  "gender": "Women",
  "department": "Shoes",
  "ec_brand": "ACME"
}
```

### Merging with existing items

To perform a shallow merge on your commerce [items](https://docs.coveo.com/en/pa8f6515/), you must interact with the [Coveo Stream API](https://docs.coveo.com/en/12#tag/Stream).
This section guides you through the different actions that must be taken to update your commerce [items](https://docs.coveo.com/en/pa8f6515/).

Refer to the [Coveo Stream API](https://docs.coveo.com/en/12#tag/Stream) for a comprehensive list of required parameters.

#### Step 1: Create a file container

> **Important**
>
> Make sure that you meet the [prerequisites](#prerequisites) before performing this operation.

To perform a merge document update, you must first [create an Amazon S3 file container](https://docs.coveo.com/en/43/).
Use the [Create a file container](https://docs.coveo.com/en/12/api-reference/push-api#tag/File-Container/paths/~1organizations~1%7BorganizationId%7D~1files/post) operation to create an Amazon S3 file container for a specific [Coveo organization](https://docs.coveo.com/en/185/):

**Request template**

```http
POST https://api.cloud.coveo.com/push/v1/organizations/<MyOrganizationId>/files?useVirtualHostedStyleUrl=<true|false> HTTP/1.1
​
Accept: application/json
Content-Type: application/json
Authorization: Bearer <MyAccessToken>
```

In the request path:

* Replace `<MyOrganizationId>` with your [organization ID](https://docs.coveo.com/en/148/).

In the query string:

* Optionally, set `useVirtualHostedStyleUrl` to `true` if you want the service to return a virtual hosted-style URL, such as `+coveo-nprod-customerdata.s3.amazonaws.com/...+`.
The default value is currently `false`, which means that the service returns path-style URLs, such as `+s3.amazonaws.com/coveo-nprod-customerdata/...+`.

> **Important**
>
> The `useVirtualHostedStyleUrl` query string parameter will soon be deprecated as part of the [path-style URL deprecation](https://docs.coveo.com/en/43#path-style-url-deprecation).
> From this point onwards, the service will only return virtual hosted-style URLs.

In the `Authorization` HTTP header:

* Replace `<MyAccessToken>` with an access token, such as an [API key](https://docs.coveo.com/en/1718/) that has the [required privileges](#required-privileges) to push content to the source.

**Payload**: None

The body of a successful response contains important information about the temporary, private, and encrypted Amazon S3 file container that you just created:

```json
{
    "uploadUri": "<UPLOAD_URI>", <1>
    "fileId": "<FILE_ID>", <2>
    "requiredHeaders": { <3>
        "x-amz-server-side-encryption": "AES256",
        "Content-Type": "application/octet-stream"
    }
}
```

<1> The `uploadUri` property contains a pre-signed URI to
use in the PUT request of [step 2](#step-2-upload-the-merge-item-content-into-the-file-container).

> **Notes**
>
> * The Amazon S3 file container applies [AES-256](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) server-side encryption to your data.
> 
> * The file container is automatically deleted as soon as its content has been successfully forwarded to the service.
> 
> * The `uploadUri` automatically expires after 60 minutes.
> 
> Therefore, it's safe to upload sensitive information into the Amazon S3 file container.

<2> The `fileId` property contains the unique identifier of your file container.
You must use this value to send the file container to the source in step 3.
<3> The `requiredHeaders` property contains the required HTTP headers for sending
in the PUT request of [step 2](#step-2-upload-the-merge-item-content-into-the-file-container).

#### Step 2: Upload the merge item content into the file container

To upload the content to merge into the [file container you created in step 1](#step-1-create-a-file-container), perform the following PUT request:

**Request template**

```http
PUT <MyUploadURI> HTTP/1.1

<HTTPHeaders>
```

Where you replace:

* `<MyUploadURI>` with the value of the `uploadUri` property from the [step 1 HTTP response](#step-1-create-a-file-container).

* `<HTTPHeaders>` with the key-value pairs of the `requiredHeaders` object property from the [step 1 HTTP response](#step-1-create-a-file-container).

You can now upload your update data in the body of the request.
You must structure the data in JSON format and upload it as either a JSON file or a file compressed with the ZLib algorithm.
For example, the following payload illustrates how you could replace metadata on existing commerce [items](https://docs.coveo.com/en/pa8f6515/):

**Payload example**

```json
{
   "addOrMerge": [ <1>
       {
           "objecttype": "Product",
           "documentId": "product://010",
           "ec_name": "Sneaker 010",
           "productId": "010",
           "ec_category": "Sneakers",
           "gender": "Unisex",
           "department": "Shoes"
       },
       {
           "objecttype": "Variant",
           "documentId": "variant://010-blue",
           "width": "wide"
       },
       // ...More items to add or merge...
   ]
}
```

In the [request body](https://docs.coveo.com/en/78#batchdocumentbody-model):

<1> For each [item](https://docs.coveo.com/en/pa8f6515/) you include in the `addOrMerge` array, you must specify a unique `documentId` value.
Therefore, make sure that all of your [items](https://docs.coveo.com/en/pa8f6515/) contain a unique `documentId` whose value is an URI.
This value must be a valid URL with a proper URI prefix, such as `product://`, or any other scheme that fits your [catalog data](https://docs.coveo.com/en/obcf0333/).

A successful response has no content, but indicates that the content update was successfully uploaded to the Amazon S3 file container, as in the following example:

**200 OK**

```json
{}
```

> **Important**
>
> When the payload exceeds 256 MB, it must be chunked into 256 MB parts.
> See [Uploading large catalog data files](#uploading-large-catalog-data-files) for instructions.

#### Step 3: Send the file container to update your catalog (full item update)

To push the Amazon S3 file container into your [source](https://docs.coveo.com/en/246/), use the [Merge documents of a catalog stream source](https://platform.cloud.coveo.com/docs?urls.primaryName=PushAPI#/Stream/put_organizations__organizationId__sources__sourceId__stream_merge) operation as follows:

**Request template**

```http
PUT https://api.cloud.coveo.com/push/v1/organizations/<MyOrganizationId>/sources/<MySourceId>/stream/merge?fileId=<MyFileId> HTTP/1.1

Content-Type: application/json
Authorization: Bearer <MyAccessToken>
```

**Payload**

```json
{}
```

In the request path:

* Replace `<MyOrganizationId>` with the ID of the target [Coveo organization](https://docs.coveo.com/en/185/) (see [Retrieve the organization ID](https://docs.coveo.com/en/148/)).

* Replace `<MySourceId>` with the [ID](https://docs.coveo.com/en/3390#copy-a-source-name-or-id) of the [source](https://docs.coveo.com/en/246/) which contains the [catalog data](https://docs.coveo.com/en/obcf0333/) that you want to merge with.

In the [query](https://docs.coveo.com/en/231/) string:

* Replace `<MyFileId>` with the `fileId` from the [step 1 HTTP response](#step-1-create-a-file-container).

In the `Authorization` HTTP header:

* Replace `<MyAccessToken>` with an access token, such as an [API key](https://docs.coveo.com/en/1718/) that has the [required privileges](#required-privileges) to push content to the source.

A successful response indicates that the operation was successfully forwarded to the service and that the batch of [items](https://docs.coveo.com/en/pa8f6515/) is now enqueued to be processed by the [Coveo indexing pipeline](https://docs.coveo.com/en/184/).
The response body contains an [`orderingId`](https://docs.coveo.com/en/147/) that indicates the time your request was received, as well as the `requestId` which is the unique identifier for your request.

**Example**

**202 Accepted**

```json
{
  "orderingId": 1716387965000,
  "requestId": "498ef728-1dc2-4b01-be5f-e8f8f1154a99"
}
```

> **Tip**
>
> The contents of a file container can be pushed to multiple sources in the same [Coveo organization](https://docs.coveo.com/en/185/).
> Just update the target `sourceId` and Authorization HTTP header access token in your other Stream API merge or [update requests](https://docs.coveo.com/en/p4eb0515/).
> 
> The file container remains available for 4 days.

### Shallow merge operations and dictionary fields

Performing a shallow merge operation on an existing [dictionary field](https://docs.coveo.com/en/2036/) replaces the entire value(s).
This means that this operation shouldn't be used to append or remove values from a dictionary field.

For example, consider the following dictionary field:

```json
{
   ...
   "documentId": "product://010",
   "price_dict": {
       "": "28.00",
       "store1": "28.00",
       "store2": "30.00"
   },
   ...
}
```

You want to update the `price_dict` field to add a new store.
You use a shallow merge operation such as the following:

```json
{
  "addOrMerge": [
    {
      "documentId": "product://010",
      "price_dict": {
        "store3": "32.00"
      }
    }
  ]
}
```

The `price_dict` field is replaced by the new value, and the existing values are lost.
The item now appears in the [source](https://docs.coveo.com/en/246/) with the following [metadata](https://docs.coveo.com/en/218/):

```json
{
   ...
   "documentId": "product://010",
   "price_dict": {
       "store3": "32.00"
   },
   ...
}
```

To append or remove values from a dictionary field, you should instead use a [partial item update](#step-1-create-a-file-container-partial-item-update) operation.

## Stream API limits

The Stream API enforces certain limits on request size and frequency.

These limits differ depending on whether the [organization](https://docs.coveo.com/en/185/) to which data is pushed is a production or [non-production](https://docs.coveo.com/en/2959/) [organization](https://docs.coveo.com/en/185/).

The following table indicates the Stream API limits depending on your [organization](https://docs.coveo.com/en/185/) type:

[%header,cols="3"]
|===
|API limits
|Production
|Non-production

|Stream API calls per day
|15,000
|10,000

|Burst limit (Stream API calls per 5 minutes)
|250
|150

|Stream API loads per day
|96
|96

|Maximum file size
|256 MB
|256 MB

|Maximum [item](https://docs.coveo.com/en/210/) size
|3 MB
|3 MB

|Total size of indexed items per day[.footnote]^[[1](#limit-note1)]^
|50 GB
|50 GB

|[Standard commerce field](https://docs.coveo.com/en/n73f0502#standard-commerce-fields) data allowance per item
|1 MB
|1 MB

|Maximum items per [source](https://docs.coveo.com/en/246/)
|2,000,000
|2,000,000

|Partial updates per day
|12,000,000
|12,000,000

|Partial updates per 5 minutes[.footnote]^[[1](#limit-note1)]^
|1,000,000
|1,000,000

|Full item updates per 5 minutes[.footnote]^[[1](#limit-note1)]^
|600,000
|600,000
|===

--
1. This limit will be applied starting January 31, 2026.
--

> **Important**
>
> These limits could change at any time without prior notice.
> You can review your Push API usage and limits on the [**System Performance**](https://platform.cloud.coveo.com/admin/#/orgid/organization/system-performance/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/organization/system-performance/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/organization/system-performance/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/organization/system-performance/)) page of the [Coveo Administration Console](https://docs.coveo.com/en/183/).
> 
> To modify these limits, contact your Coveo representative.

## Stream API error codes

If a request to the Stream API fails because one of the limits has been exceeded, the API will trigger one of the following response status codes:

[%header,cols="1,5"]
|===
|Status code
|Triggered when

|[`413`](https://docs.coveo.com/en/95#413-request-entity-too-large)
|The total Stream API request size exceeds 256 MB when pushing a large file container.
See [Uploading large catalog files](#uploading-large-catalog-data-files).

.4+|[`429`](https://docs.coveo.com/en/95#429)
|The amount of total Stream API (upload and update) requests exceeds 15,000 per day (10,000 for [non-production organizations](https://docs.coveo.com/en/2959/)).
The quota is reset at midnight [UTC](https://en.wikipedia.org/wiki/Coordinated_Universal_Time).
|The amount of total Stream API upload requests exceeds 96 per day (4 per hour).
The quota is reset at midnight [UTC](https://en.wikipedia.org/wiki/Coordinated_Universal_Time).
|The amount of total Stream API requests exceeds 250 (150 for [non-production organizations](https://docs.coveo.com/en/2959/)) within a 5 minute period.
The [`retry-after`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) header indicates how long the user agent should wait before making another request.
|[Coveo declined your request due to a reduced indexing capacity](https://docs.coveo.com/en/n91c0225/).
|===

You can review your Push API usage and limits on the [**System Performance**](https://platform.cloud.coveo.com/admin/#/orgid/organization/system-performance/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/organization/system-performance/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/organization/system-performance/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/organization/system-performance/)) page of the [Coveo Administration Console](https://docs.coveo.com/en/183/).

:leveloffset!:

## Uploading large catalog data files

The Stream API limits the size of your [catalog data](https://docs.coveo.com/en/obcf0333/) JSON file to 256 MB.
If your catalog data file exceeds the limit, you must upload multiple JSON files.

To upload multiple JSON files:

When you initially open the stream, you receive an `uploadUri`.
This URI is used to upload your first set of metadata (JSON file).

. After uploading the first file, make a POST request to the following endpoint to get a new `uploadUri`:

```http
POST https://api.cloud.coveo.com/push/v1/organizations/{organizationId}/sources/{sourceId}/stream/{streamId}/chunk HTTP/1.1

Content-Type: application/json
Accept: application/json
Authorization: Bearer <MY_ACCESS_TOKEN>
```

This request returns a new `uploadUri` that you can use for the next step.

. Make a PUT request using the `uploadUri` your received in the previous step.
The body of the request must contain the [catalog data](https://docs.coveo.com/en/obcf0333/) chunk (maximum 256 MB) that you want to upload.

```http
PUT {uploadUri} HTTP/1.1

x-amz-server-side-encryption: AES256
Content-Type: application/octet-stream
```

If your request to upload the catalog data is successful, you'll receive a `200` HTTP response code.

. If you have more [catalog data](https://docs.coveo.com/en/obcf0333/) files to upload, repeat this process until all of your [catalog data](https://docs.coveo.com/en/obcf0333/) has been uploaded.
For each file, first obtain a new `uploadUri`, then upload the file.

## Troubleshooting

## `NO_VALID_OPERATION_IN_BATCH` error

When using the [Coveo Stream API](https://docs.coveo.com/en/12#tag/Stream) to perform partial updates on [catalog data](https://docs.coveo.com/en/obcf0333/), you may encounter the `NO_VALID_OPERATION_IN_BATCH` error in the [**Log Browser**](https://platform.cloud.coveo.com/admin/#/orgid/logs/browser/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/logs/browser/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/logs/browser/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/logs/browser/)).

### Common causes

When performing partial updates on [catalog data](https://docs.coveo.com/en/obcf0333/), this error occurs if a valid [catalog entity](https://docs.coveo.com/en/3143/) isn't properly associated with the [source](https://docs.coveo.com/en/246/).
Without the correct [catalog entity](https://docs.coveo.com/en/3143/) setup, updates may be rejected even if the payload appears valid.

### Solution

**Verify catalog prerequisites**: Confirm that you've created and configured a [catalog entity](https://docs.coveo.com/en/3143/) for your [source](https://docs.coveo.com/en/246/).
See [Commerce catalog entity](https://docs.coveo.com/en/3139/) for details.

:leveloffset!:

## Required privileges

The following table indicates the [privileges](https://docs.coveo.com/en/228/) required for your [organization](https://docs.coveo.com/en/185/)’s [groups](https://docs.coveo.com/en/2867/) to view or edit elements of the [**Catalogs**](https://platform.cloud.coveo.com/admin/#/orgid/commerce/catalogs/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/commerce/catalogs/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/commerce/catalogs/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/commerce/catalogs/)) page and its associated panels (see [Manage privileges](https://docs.coveo.com/en/3151/) and [Privilege reference](https://docs.coveo.com/en/1707/)).
The **Commerce** domain is only available to [organizations](https://docs.coveo.com/en/185/) in which Coveo for Commerce features are enabled.
[cols="3",options="header"]
|===
|Action
|Service - Domain
|Required access level

|View catalogs
|Commerce - Catalogs  
Content - Sources  
Content - Fields  
Organization - Organization
|View

.3+.^|Edit catalogs
|Content - Fields  
Content - Sources  
Organization - Organization
|View

|Commerce - Catalogs
|Edit

|Search - Execute Query
|Allowed
|===