Using the compressedBinaryDataFileId Property

This is for:

Developer System Administrator

Use the compressedBinaryDataFileId property when you want to push 256 MB or less of compressed or uncompressed, binary or non-binary item data.

Using this property to push item data involves three distinct steps which are detailed in this article.

Step 1: Create a File Container

The first thing you need to do is to create a file container.

Step 2: Upload the Item Data Into the File Container

Next, you need to perform the following PUT request to upload your item data into the Amazon S3 file container you got from Step 1.

Request template

PUT <MyUploadURI> HTTP/1.1
​
<HTTPHeaders>
​
<MyItemData>

Replace:

  • <MyUploadURI> with the uploadUri you got from Step 1.

  • <HTTPHeaders> with the requiredHeaders you got from Step 1.

  • <MyItemData> with your uncompressed or Deflate, GZip, LZMA, or ZLib compressed item data. Your request body must be in binary format.

    Don’t Base64 encode your item data.

Sample Request

Uploading Deflate compressed item data

PUT https://coveo-nprod-customerdata.s3.amazonaws.com/proda/blobstore/mycoveocloudv2organizationg8tp8wu3/05ee7647-15a6-464a-8ffd-cd2fb6c1a03b[...] HTTP/1.1
​
x-amz-server-side-encryption: AES256
Content-Type: application/octet-stream

Payload (binary format)

eJyzySjJzbHjUgACm4zUxBQoUx+JnZSfUglhQlQZ2rlWJOYW5KQCVRkiSRTYhWRkFisAUWKeQipEiZ6NfoEdF6eNPsQQoA6QdQDMhB8E

Successful response - 200 OK

{}

Step 3: Use the fileId as the CompressedBinaryDataFileId Value

Once the request you performed in Step 2 has successfully returned, you can use the fileId you got from Step 1 as the compressedBinaryDataFileId property value.

Using the compressedBinaryDataFileId Property when Pushing a Single Item

The following example shows how you can use the compressedBinaryDataFileId property to specify item data when performing a single push operation (see Add or Update a Single Item in a Push Source).

When using the compressedBinaryDataFileId property to push a single item, your query string should include the compressionType parameter along with the compression algorithm that was applied on the data:

  • UNCOMPRESSED

  • DEFLATE

  • GZIP

  • LZMA

  • ZLIB

Those values are case sensitive. The default value is ZLIB.

Sample request

PUT https://api.cloud.coveo.com/push/v1/organizations/mycoveocloudv2organizationg8tp8wu3/sources/mycoveocloudv2organization-rp5rxzbdz753uhndklv2ztkfgy/documents?documentId=file://example.html&compressionType=DEFLATE HTTP/1.1
​
Content-Type: application/json
Accept: application/json
Authorization: Bearer **********-****-****-****-************

Payload

{
  // ...Item metadata...
  "compressedBinaryDataFileId": "b5e8767e-8f0d-4a89-9095-1127915c89c7",
  "fileExtension": ".html",
  // ...Item permission model (if applicable)...
}

Successful response - 202 Accepted

{}

Using the compressedBinaryDataFileId Property when Uploading a Batch of Items

Using the compressedBinaryDataFileId property to upload a batch of items involves more steps.

Step 3.1: Create a File Container

Create a new Amazon S3 file container, this time for your batch of items.

Step 3.2: Upload the Batch of Items Using the compressedBinaryDataFileId Property

Upload your batch of items into your newly created file container. In the target item metadata, use the CompressedBinaryDataFileId property to refer to the file container in which you uploaded item data at Step 2.

When using the compressedBinaryDataFileId property to push the data of an item in a batch, the document body this item should include the compressionType property along with the compression algorithm that was applied on the data:

  • "UNCOMPRESSED"

  • "DEFLATE"

  • "GZIP"

  • "LZMA"

  • "ZLIB"

These values are case sensitive.

The default value is "ZLIB".

Uploading a batch of items to a file container, referring to an uploaded compressedBinaryDataFileId in an item

PUT https://coveo-nprod-customerdata.s3.amazonaws.com/proda/blobstore/mycoveocloudv2organizationg8tp8wu3/8d3jf7sh-8d3h-sd89-d1s2-d9fj2ls9dnv7[...] HTTP/1.1
​
x-amz-server-side-encryption: AES256
Content-Type: application/octet-stream

Payload

{
  "addOrUpdate": [
    {
      // ...Item metadata...
      "documentId": "file://example.html",
      "compressedBinaryDataFileId": "b5e8767e-8f0d-4a89-9095-1127915c89c7",
      "compressionType": "DEFLATE",
      "fileExtension": ".html",
      // ...Item permission model (if applicable)...
    },
    // ...More items to add or update...
  ],
  "delete": [
    // ...Items to delete...
  ]
}

Successful response - 200 OK

{}

Step 3.3: Push the File Container Into Your Push Source

Take the file container in which you just uploaded your batch of items, and push it into your source.