--- title: Uploading a data stream for an extension test slug: '2916' canonical_url: https://docs.coveo.com/en/2916/ collection: index-content source_format: adoc --- # Uploading a data stream for an extension test When using the Extension API [`/test`](https://platform.cloud.coveo.com/docs?urls.primaryName=Extension#/Indexing%20Pipeline%20Extensions/rest_organizations_paramId_extensions_paramId_test_post) endpoint to test an [indexing pipeline extension (IPE)](https://docs.coveo.com/en/206/), you can pass [data streams](https://docs.coveo.com/en/2891/) in the mock documents of your request bodies. If those data streams are very large, it can be impractical to pass them as inline content. This is why the Extension API exposes the [`/test/datastream`](https://platform.cloud.coveo.com/docs?urls.primaryName=Extension#/Indexing%20Pipeline%20Extensions/rest_organizations_paramId_extensions_paramId_test_datastream_post) endpoint, which allows you to retrieve URIs where you can upload data streams. You can then refer to those uploaded data streams by their id in the mock document of your `/test` request body. This article provides more detailed explanations of those steps. ## Step 1 (optional): Request a data stream The Search API exposes endpoints to retrieve the [thumbnails](https://platform.cloud.coveo.com/docs?urls.primaryName=Search%20API#/Search%20V2/dataStream), [text bodies](https://platform.cloud.coveo.com/docs?urls.primaryName=Search%20API#/Search%20V2/text), and [HTML bodies](https://platform.cloud.coveo.com/docs?urls.primaryName=Search%20API#/Search%20V2/htmlGet) of [indexed](https://docs.coveo.com/en/204/) [items](https://docs.coveo.com/en/210/) (To carry out a test on a `documentdata` stream, you can use the original item itself.). Be aware that those data streams may not match exactly the ones which would actually be available when executing an IPE. Nonetheless, they're typically similar enough to carry out meaningful tests. You can alternatively craft your own data streams. Authenticate the request using an access token granting the **Search - Execute queries - Allowed** [privilege](https://docs.coveo.com/en/228/) (see [Manage API keys](https://docs.coveo.com/en/1718/)). Retrieve and store the body of a successful response. **Example** Using the Search API to retrieve the HTML body of a specific YouTube item: ```http GET https://platform.cloud.coveo.com/rest/search/v2/html?organizationId=myorganization&uniqueId=42.1294$https://youtube.com/User:TheLumineers/Channel:UCB7P9Hr5BYB5Mkxau6t3Sgw/Video:zND5Qc7LzdM HTTP/1.1 Authorization: Bearer **********-****-****-****-************ ``` **200 OK:** ```xml

The Lumineers performing "Stubborn Love" at the Tractor Tavern in Seattle - 9.27.2011

Video taken by Gerad Simon.

http://www.thelumineers.com/
http://www.facebook.com/TheLumineers
http://www.myspace.com/thelumineers

June 04 -- Cincinnati, OH @ Riverbend Music Center*
June 06 -- Clarkston, MI @ DTE Energy Music Theatre*%
June 07 -- Canandaugia, NY @ CMAC*%
June 08 -- Mountain Hunter, NY @ Mountain Jam Festival
June 09 -- New York, NY @ Governor's Ball
June 12 -- Portsmouth, VA @ nTelos Wireless Pavilion*%
June 13 -- Cary, NC @ Koka Booth Amphitheater*% (Presale: April 10; On-sale April 12)
June 14 -- Charleston, SC @ Family Circle Pavilion* (Presale: April 10; On-sale April 12)
June 15 -- Manchester, TN @ Bonnaroo Music Festival
June 23 -- Dover, DE @ Firefly Music Festival
June 28 -- Worthy Farm, Pilton @ Glastonbury
June 29 -- Paris @ L'Olympia
June 30 -- St. Gallen @ Open Air St. Gallen
July 05 -- Werchter @ Rock Werchter 2013
July 08 -- Montreux @ Montreux Jazz Festival
July 09 -- Milan @ Un Altro Music Festival
July 10 -- Barcelona @ Razzmatazz
July 13 -- Balado, Kinnross-Shire, Scotland @ T in the Park
July 25 -- Floyd, VA @ FloydFest
July 26 -- Columbia, MD @ Merriweather Post Pavilion
July 27 -- Camden, NJ @ XPoNential Music Festival
July 28 -- Newport, RI @ Newport Folk Festival
July 31 -- Toronto, ON @ Edgefest '13
Aug 02 -- Chicago, IL @ Lollapalooza
Aug 04 -- Montreal, QC @ Osheaga Music & Arts Festival
Aug 16 -- Hamburg @ Dockville Festival
Aug 17 -- Biddinghuizen @ Lowlands
Aug 18 -- Leipzig @ Highfield Festival
Aug 19 -- Hanau @ Amphitheater Hanau
Aug 22 -- Belfast @ Belsonic 2013 // Custom House Square
Aug 23 -- Leeds @ Reading Festival
Sept 14 -- Colorado @ Red Rocks Amphitheatre
Sept 15 -- Colorado @ Red Rocks Amphitheatre
Nov 17 -- Paris @ Zenith
Nov 18 -- Amsterdam @ Heineken Music Hall
Nov 21 -- Dublin @ 02 Arena
Dec 01 -- Brussels @ Cirque Royal
Dec 02 -- Winterthur @Eishalle Deutweg
Dec 03 -- Bolgna @ Estragon
Dec 06 -- Munich @ Zenith
Dec 07 -- Berlin @ Columbiahalle
Dec 08 -- Cologne @ Palladium
Dec 10 -- Copenhagen @ Vega
Dec 11 -- Oslo @ Sentrum Scene
Dec 12 -- Stockholm @ Annexet

*with Cold War Kids
%with J Roddy Walston

``` ## Step 2: Request an upload URI Use the [`+POST /rest/organizations/{organizationId}/extensions/{extensionId}/test/datastream+`](https://platform.cloud.coveo.com/docs?urls.primaryName=Extension#/Indexing%20Pipeline%20Extensions/rest_organizations_paramId_extensions_paramId_test_datastream_post) operation to request an upload URI, specifying your `organizationId` and `extensionId` path parameters. Authenticate the request using an access token granting the **Content - Extension - Edit** privilege on the target extension (see [Manage API keys](https://docs.coveo.com/en/1718/)). In the body of a successful response, retrieve and store the `streamId`, the `uri`, and the `requiredHeaders` properties. **Example** ```http POST https://platform.cloud.coveo.com/rest/organizations/myorganization/extensions/myorganization-vwlyqfbtjkotxqkmrxqjcbcpoy/test/datastream HTTP/1.1 Authorization: Bearer **********-****-****-****-************ ``` **200 OK** ```json { "streamId": "73e67b38-c307-429e-ac89-147083f31ca2", "uri": "link:https://coveo-nprod-customerdata.s3.amazonaws.com/proda/blobstore/myorganization/73e67b38-c307-429e-ac89-147083f31ca2?X-Amz-Security-Token[...]", "requiredHeaders": { "x-amz-server-side-encryption": "AES256", "Content-Type": "application/octet-stream" } } ``` ## Step 3: Upload a data stream Make a PUT request to the `uri` obtained in step 1, using the `requiredHeaders` also obtained in step 1, and where the body of your call is your data stream, in raw format. If you requested a data stream from the Search API in [Step 1](#step-1-optional-request-a-data-stream), this is where you use it. > **Note** > > To use your uploaded data stream as the `BODY_TEXT` of your mock document, you will likely want your request body to be a UTF-16LE encoded binary file, since this is the encoding expected by the Extension API in such cases. **Example** ```http PUT link:https://coveo-nprod-customerdata.s3.amazonaws.com/proda/blobstore/myorganization/73e67b38-c307-429e-ac89-147083f31ca2?X-Amz-Security-Token[...] HTTP/1.1 x-amz-server-side-encryption: AES256 Content-Type: application/octet-stream ``` Request body: ```xml

``` ## Step 4: Use the data stream in an extension test Use the [`+POST /rest/organizations/{organizationId}/extensions/{extensionId}/test+`](https://platform.cloud.coveo.com/docs?urls.primaryName=Extension#/Indexing%20Pipeline%20Extensions/rest_organizations_paramId_extensions_paramId_test_post) operation to test your extension, specifying your `organizationId` and `extensionId` path parameters. In the request body, set the `id` of the target data stream to the `streamId` value retrieved in step 2. The possible data stream values are `DOCUMENT_DATA`, `BODY_HTML`, `BODY_TEXT`, and `THUMBNAIL`. Authenticate the request using an access token granting the **Content - Extension - Edit** [privilege](https://docs.coveo.com/en/228/) on the target extension (see [Manage API keys](https://docs.coveo.com/en/1718/)). **Example** You have a simple extension which logs the HTML body of documents. ```python myDataStream = document.get_data_stream('body_html').read() log(myDataStream) ``` You use the data stream uploaded in step 3 to test your extension. ```http POST https://platform.cloud.coveo.com/rest/organizations/myorganization/extensions/myorganization-vwlyqfbtjkotxqkmrxqjcbcpoy/test HTTP/1.1 Authorization: Bearer **********-****-****-****-************ ``` Request body: ```json { "document":{ "dataStreams":[ { "values":{ "BODY_HTML":{ "id":"73e67b38-c307-429e-ac89-147083f31ca2" } } } ] } } ``` **200 OK** response body (excerpt): ```json { ... "logs": [ { "severity": "NORMAL", "comment": "\n\n\n\n\n\t

\n\t\t

\n\t\t\t\n\t\t\t\n\t\t

\n\t\t

\n\t\t\t

The Lumineers performing \"Stubborn Love\" at the Tractor Tavern in Seattle -\n\t\t\t\t9.27.2011

Video taken by Gerad Simon.

http://www.thelumineers.com/
http://www.facebook.com/TheLumineers
http://www.myspace.com/thelumineers

June 04 -- Cincinnati, OH @ Riverbend Music Center*
June 06 -- Clarkston, MI @ DTE Energy Music Theatre*%
June 07 -- Canandaugia, NY @ CMAC*%
June 08 -- Mountain Hunter, NY @ Mountain Jam Festival
June 09 -- New York, NY @ Governor's Ball
June 12 -- Portsmouth, VA @ nTelos Wireless Pavilion*%
June 13 -- Cary, NC @ Koka Booth Amphitheater*% (Presale: April 10; On-sale April 12)
June 14 -- Charleston, SC @ Family Circle Pavilion* (Presale: April 10; On-sale April 12)
June 15 -- Manchester, TN @ Bonnaroo Music Festival
June 23 -- Dover, DE @ Firefly Music Festival
June 28 -- Worthy Farm, Pilton @ Glastonbury
June 29 -- Paris @ L'Olympia
June 30 -- St. Gallen @ Open Air St. Gallen
July 05 -- Werchter @ Rock Werchter 2013
July 08 -- Montreux @ Montreux Jazz Festival
July 09 -- Milan @ Un Altro Music Festival
July 10 -- Barcelona @ Razzmatazz
July 13 -- Balado, Kinnross-Shire, Scotland @ T in the Park
July 25 -- Floyd, VA @ FloydFest\n\t\t\t\t
July 26 -- Columbia, MD @ Merriweather Post Pavilion
July 27 -- Camden, NJ @ XPoNential Music Festival
July 28 -- Newport, RI @ Newport Folk Festival
July 31 -- Toronto, ON @ Edgefest '13
Aug 02 -- Chicago, IL @ Lollapalooza
Aug 04 -- Montreal, QC @ Osheaga Music & Arts Festival
Aug 16 -- Hamburg @ Dockville Festival
Aug 17 -- Biddinghuizen @ Lowlands
Aug 18 -- Leipzig @ Highfield Festival
Aug 19 -- Hanau @ Amphitheater Hanau\n\t\t\t\t
Aug 22 -- Belfast @ Belsonic 2013 // Custom House Square
Aug 23 -- Leeds @ Reading Festival
Sept 14 -- Colorado @ Red Rocks Amphitheatre
Sept 15 -- Colorado @ Red Rocks Amphitheatre
Nov 17 -- Paris @ Zenith
Nov 18 -- Amsterdam @ Heineken Music Hall
Nov 21 -- Dublin @ 02 Arena
Dec 01 -- Brussels @ Cirque Royal
Dec 02 -- Winterthur @Eishalle Deutweg
Dec 03 -- Bolgna @ Estragon
Dec 06 -- Munich @ Zenith
Dec 07 -- Berlin @ Columbiahalle
Dec 08 -- Cologne @ Palladium\n\t\t\t\t
Dec 10 -- Copenhagen @ Vega
Dec 11 -- Oslo @ Sentrum Scene
Dec 12 -- Stockholm @ Annexet

*with Cold War Kids
%with J Roddy Walston

\n\t\t

\n\t

\n\n\n" } ], ... "duration": 0.103328537, "result": { "status": "SUCCESS" } } ``` In the above response body, you can see that the `comment` property is indeed the content of your uploaded data stream, confirming your extension works as expected.