REST API source reference
REST API source reference
When creating a REST API source in the Coveo Administration Console, you must provide a JSON configuration detailing which services and endpoints to crawl to retrieve the desired content, and how to retrieve items of each type.
This configuration has a single property, Services
, whose value must be an array of objects representing the endpoints to crawl.
This reference article defines parameters to include in your JSON configuration.
When working on your REST API source, you may also want to refer to the following articles:
In addition, you may want to use the table of contents to quickly jump to the right property.
Example
The following is a basic example of a REST API source configuration. It instructs Coveo to call an endpoint to retrieve a list of users.
Basic example
If you expect your API to return the following response:
[
{ "id": 1, "name": "Caroline", "updated": "2023-11-14 11:29:49" },
{ "id": 2, "name": "Marcella", "updated": "2023-11-15 13:51:24" },
{ "id": 3, "name": "Susie", "updated": "2023-11-15 13:59:44" },
{ "id": 4, "name": "Rhonda", "updated": "2023-11-15 16:09:08" },
{ "id": 5, "name": "Wendy", "updated": "2023-11-16 17:02:35" }
{ "id": 6, "name": "Barbara", "updated": "2023-11-16 17:02:35" }
{ "id": 7, "name": "Deirdre", "updated": "2023-11-16 17:02:35" }
{ "id": 8, "name": "Lynda", "updated": "2023-11-16 17:02:35" }
]
Then your JSON configuration could look like this:
{
"Services": [
{
"Url": "http://example.com/api/v1",
"Headers": {
"Authorization": "Bearer @ApiKey"
},
"Endpoints": [
{
"Path": "/users",
"Method": "GET",
"ItemType": "User",
"Uri": "%[coveo_url]/users/%[id]",
"ClickableUri": "%[coveo_url]/users/%[id]",
"Title": "%[name]",
"ModifiedDate": "%[updated]",
"Metadata": {
"id": "%[id]",
"username": "%[name]"
}
}
]
}
]
}
For additional, more complex examples, see JSON configuration examples. You can also use the table of contents to jump to a specific parameter.
Table of contents
To help you jump to the right section, here’s a hierarchical and alphabetical list of all supported arrays, objects, and parameters.
Show table of contents
Hierarchical list | Alphabetical list |
---|---|
|
|
Services
(array, required)
Each object in the Services
array specifies a REST API service to retrieve content, along with the specific endpoints to call.
The properties that may appear in an object of the Services
array are the following:
Endpoints
(array, required)
Services
> Endpoints
Each object in the Endpoints
array details how you want Coveo to crawl an endpoint of your web application.
Moreover, in each endpoint object, the Metadata
objects lists the content metadata to retrieve.
Objects of the Endpoints
array support the following properties from their parent service object:
It also supports the following properties:
-
ClickableUri
(required) -
ItemType
(required) -
Path
(required) -
Uri
(required)
ClickableUri
(string, required)
The format of the item clickable URI.
Dynamic values are supported.
Example: "ClickableUri": "%[coveo_url]/posts/%[id]"
ItemType
(string, required)
The type of item that the resource contains. Coveo uses this data to organize the indexed content.
Examples:
-
"ItemType": "Post"
for a blog post -
"ItemType": "PostComment"
for a comment on a post
Path
(string, required)
The relative path to the desired resource endpoint.
Dynamic values are supported when specified within the SubItems
, SubQueries
, and PermissionSubQueries
arrays only.
Dynamic time expressions are also supported.
Example: "Path": "orgs/myorg/repos"
retrieves your GitHub repositories
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.
Uri
(string, required)
The format of the item unique identifier.
Dynamic values are supported.
Example: "Uri": "%[coveo_url]/repository/posts/%[id]"
Should a dynamic value be invalid or unretrievable, the source stops crawling and displays an error in the Administration Console.
Body
(string)
The path (simple path or JSONPath) to the Body
content of the item in the JSON response.
The item body is displayed in the Content Browser.
Typically, the Body
property contains a dynamic value, since a static, hardcoded value would make all item bodies identical.
Dynamic values and hardcoded text can also be mixed.
Example: "Body": "%[html.content]"
DateFormat
(string)
The format of a date injected thanks to the tokens @Now
and @RefreshDate
.
All .NET formats are supported, and the default is yyyy-MM-ddTHH:mm:ssZ
(ISO 8601).
Alternatively, to handle Unix timestamps, enter either of the following:
-
UnixEpoch
for the number of seconds elapsed since the Unix epoch. -
UnixEpochMs
for the number of milliseconds elapsed since the Unix epoch.
Coveo also uses this format when parsing dates coming from your API.
If these dates aren’t in a .NET format, use DateFormat
to specify the format to expect.
Examples:
-
"DateFormat": "yyyy-MM-ddTHH:mm:ssZ"
-
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'"
IndexingAction
(object)
Whether to ignore or to retrieve an item when a condition is met.
When the Condition
resolves to true, the Coveo crawler applies the specified action to the item.
Possible actions are Retrieve
and Ignore
, that is, the crawler can either index the item or ignore it.
An item ignored with IndexingAction
isn’t indexed and therefore not visible in the Content Browser or a search interface.
However, its sub-items are sent to the index, unless otherwise specified in the ProcessingAction
object.
If the condition resolves to false, the Coveo crawler falls back to the default action, which is Retrieve
, and the specified item is indexed.
Example:
With the following IndexingAction
object, all items with an id
greater than 5
are ignored.
Only items with an id
of 5
or less are indexed.
"IndexingAction": {
"ActionOnItem": "Ignore",
"Condition": "%[id] > 5"
}
The IndexingAction
object supports the following properties:
-
ActionOnItem
(required)
ActionOnItem
(string, required)
Services
> Endpoints
> IndexingAction
> ActionOnItem
Action to perform if the specified Condition
resolves to true.
In the IndexingAction
object, this applies to retrieved items.
Possible actions are Retrieve
and Ignore
, that is, the crawler can either index the item to make it searchable or ignore it.
In the ProcessingAction
object, this applies to child items of an item.
Possible actions are Process
and Ignore
, that is, the crawler can either retrieve the subitems to make them searchable or ignore them.
Action Condition
(string)
Services
> Endpoints
> IndexingAction
> Condition
The Condition
value is a condition that must resolve to true for the specified action to apply to the child items of an item.
To write your condition, use the same syntax as for indexing pipeline extension conditions.
Conditions can be assembled using the following operators: AND
, OR
, Exists
, NOT
, ==
, >
, and <
.
Parentheses are also supported to specify operation order.
|
The |
The metadata fields in your condition must either be defined in the Metadata
object or referenced with raw
.
You can specify an array if the metadata refers to a multi-value field in your repository.
Examples:
-
"%[author_id]"
is true if the item has anauthor_id
. -
"NOT %[author_id]"
is true if the item doesn’t have anauthor_id
. -
"%[author_id] == 1234"
is true if the itemauthor_id
is1234
. -
"%[author_ids] == [1,2,3,4]"
is true if itemauthor_ids
are1
,2
,3
, and4
. -
"%[author_id] OR %[author_name]"
is true if the item has anauthor_id
or anauthor_name
. -
"%[author_id] AND %[author_name]"
is true if the item has anauthor_id
and anauthor_name
. -
"%[author_id] > 123"
is true if the itemauthor_id
is greater than 123. -
"(%[author_id] OR %[author_sys_id]) AND %[author_name]"
is true if the item has anauthor_id
or anauthor_name
, as well as anauthor_name
.
IsBinaryBody
(Boolean)
Whether the subquery retrieves binary content as the body of the item.
Default value is false
.
This property is available within SubQueries
objects only.
It allows you to download a file (for example, XLS or PDF document) to use as the item body in a search result Quick view.
See Supported file formats for an exhaustive list of item types that Coveo can index.
Example:
"SubQueries": [
{
"Path":"%[coveo_parent.url]",
"Method":"GET",
"IsBinaryBody":true,
"Body": "%[data.body]"
}
]
|
Notes
|
ItemPath
(string)
To index items from your application’s JSON response, Coveo needs to know where to find the items in the response.
If the items aren’t at the root level of the JSON response, you must specify the path to the items using the ItemPath
property, with JSONPath syntax.
By default, Coveo assumes that the items to index are at the root level of the JSON response.
In such case, you can omit ItemPath
.
|
Note
|
Example:
If your API’s response looks like this:
{
"count": 8,
"entries": [
{
"results": [
{ "id": 1, "name": "Caroline" },
{ "id": 2, "name": "Marcella" },
{ "id": 3, "name": "Susie" },
{ "id": 4, "name": "Rhonda" },
{ "id": 5, "name": "Wendy" },
{ "id": 6, "name": "Barbara Ann" },
{ "id": 7, "name": "Deirdre" },
{ "id": 8, "name": "Lynda" }
]
}
]
}
Then your source configuration should contain the following:
"ItemPath": "entries[0].results"
This tells Coveo to index the results
array of the first entries
object in the JSON response.
Metadata
(object)
A key-value list of the metadata to parse.
Each key represents the metadata name of the item, while its value is the value path (simple path or JSONPath) in the JSON response.
Typically, the value path consists of one or more dynamic values, since a static, hardcoded value would result in an identical piece of metadata for all items. However, you could choose to use a hardcoded value so that the corresponding Coveo field is filled even if the API doesn’t provide this information.
When you build your source, Coveo retrieves the desired metadata. You can then review a summary of this metadata in the Administration Console and use it to create mapping rules for your source.
Example
If your API returns a response with a single item that looks like this:
{
"id": 1,
"firstname": "Lloyd",
"lastname": "Sweeney",
"age": 26,
"address": {
"streetAddress": "6974 Rutrum Dr",
"city": "Ashburn",
"state": "VA",
"zip": "25581"
},
"email": "JWeakliem@lectus.org",
"username": "GHohmann",
"phone": "9556676434",
"created": "2023-09-12T05:29:48.688Z"
}
Then your source configuration could look like this:
{
"Services": [
{
"Url": "http://example.com/api/v1",
"Endpoints": [
{
"Path": "/users/1",
"Method": "GET",
"ItemType": "User",
"Uri": "%[coveo_url]/users/%[id]",
"ClickableUri": "%[coveo_url]/users/%[id]",
"Title": "%[username]",
"ModifiedDate": "%[created]",
"Metadata": {
"id": "%[id]",
"title": "Mr.",
"firstname": "%[firstname]",
"lastname": "%[lastname]",
"age": "%[age]",
"address": "%[address]",
"email": "%[email]",
"phonenumber": "%[phone]"
}
}
]
}
]
}
The title metadata is hardcoded to "Mr." for all items. |
|
In the configuration above, the address metadata contains the whole JSON object.
So, to flatten the metadata, you could also write the following Metadata object: |
{
"Metadata": {
"id": "%[id]",
"firstname": "%[firstname]",
"lastname": "%[lastname]",
"age": "%[age]",
"address.street": "%[address.streetAddress]",
"address.city": "%[address.city]",
"address.state": "%[address.state]",
"address.zip": "%[address.zip]",
"email": "%[email]",
"phonenumber": "%[phone]"
}
}
Method
(string enum)
The HTTP method to use to fetch the resource.
Possible values are GET
and POST
.
Default value is GET
.
This property is not inheritable.
See the API’s documentation to determine which one you should use.
Example: "Method": "POST"
ModifiedDate
(string)
The date on which the item was last modified. Although this property isn’t required in the JSON configuration, we recommend that you include it, as the Coveo-powered search pages use it for their default Sort by date option.
Typically, ModifiedDate
has a dynamic value, since a static, hardcoded value would make dates identical for all items.
Example: "ModifiedDate": "%[updated]"
PayloadJsonContent
(string)
The JSON content to send as a POST request body.
The Content-Type header is automatically set to application/json
.
Dynamic values are supported when specified within the SubItems
and SubQueries
objects only.
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.
|
Notes
|
You want to provide the following JSON content with your request:
{
"key1": "value1",
"key2": "@field=(\"value2\")",
"key3":
{
"subkey1": "subvalue1"
}
}
Your source JSON configuration therefore contains the following:
"Endpoints": [
{
"Method": "POST",
"Path": "/rest/myendpoint/",
"PayloadJsonContent": "{\"key1\": \"value1\", \"key2\": \"@field=(\\\"value2\\\")\", \"key3\": {\"subkey1\": \"subvalue1\"}}"
}
]
PayloadParameters
(object)
Endpoints
> PayloadParameters
A key-value list of HTTP parameters to add to the payload.
Each value can be either a number, string, or Boolean.
This property can only be used in a POST
request.
The parameters are sent as URL-encoded data in the request body.
The Content-Type header is automatically set to application/x-www-form-urlencoded
.
Dynamic values are supported when specified within the SubItems
and SubQueries
objects only.
Dynamic time expressions are also supported.
|
Notes
|
Examples:
Basic example
"PayloadParameters": {
"type": "post",
"expand": "true",
"id": 120
}
Authentication with an API key as a payload parameter
If your API requires Coveo to authenticate with an API key as the value of the api_key
payload parameter, you must enter this key in the Authentication section of the Add a REST API source panel.
![API key in the source panel](/en/assets/images/index-content/source-ref-apikeyplaceholder.png)
Then, in your source JSON configuration, you must use the @ApiKey
placeholder to refer to this key.
This ensures that your API key isn’t stored in clear text in the source JSON configuration, where other Administration Console users could see it.
When Coveo processes the source configuration, it replaces the placeholder with the actual key you entered in the panel.
Your source configuration could therefore look like this.
{
"Services": [
{
"Url": "http://example.com/api/v1",
"Endpoints": [
{
"Path": "/users",
"Method": "GET",
"ItemType": "User",
"Uri": "%[coveo_url]/users/%[id]",
"ClickableUri": "%[coveo_url]/users/%[id]",
"PayloadParameters": {
"api_key": "@ApiKey"
}
}
]
}
]
}
api_key is the name of the payload parameter that your API expects to contain the API key. |
As a result, Coveo sends the following HTTP request to the API:
POST /users HTTP/1.1
Host: http://example.com/api/v1
api_key=Az4adfSyBhZX8fTThUJ4Kb2TC3Ax0RshgrZtUiqM ![1](/en/assets/icons/callouts/1.svg)
Az4adfSyBhZX8fTThUJ4Kb2TC3Ax0RshgrZtUiqM is the API key you entered in the source panel. |
PermanentId
(string)
The PermanentId
is an alternative, permanent unique item identifier.
When you expect the item Uri
to change, this ensures that Coveo Machine Learning (Coveo ML) recognizes the items despite their new URI and associates them to the old data.
So, in the future, if you modify or delete this source and reindex the same items, specifying the same item PermanentId
format in your next configuration will ensure that ML doesn’t consider your items as new and rather picks up where it left off.
Dynamic values are supported.
Example:
{
"Uri": "%[coveo_url]/items/%[sys_id]",
"PermanentId": "%[sys_id]"
}
ProcessingAction
(object)
This object works similarly to the IndexingAction
object.
When the Condition
resolves to true, the Coveo crawler applies the specified action on the corresponding child items of an item.
Possible actions are Process
and Ignore
, that is, the crawler can either index the subitems or ignore them.
A child item ignored with ProcessingAction
isn’t indexed and therefore not visible in the Content Browser or a search interface.
If the condition resolves to false, it fallbacks to the default action, that is, Process
.
Example:
"ProcessingAction": {
"ActionOnItem": "Ignore",
"Condition": "%[id]==1"
}
Like the IndexingAction
object, the ProcessingAction
object supports the following properties:
-
ActionOnItem
(required)
QueryParameters
(object)
A key-value list of HTTP parameters to add to the query.
Each value can be either a number, string, or Boolean.
Dynamic values are supported when specified within the SubItems
, SubQueries
, and PermissionSubQueries
arrays only.
They aren’t supported in QueryParameters
objects of a refresh endpoint.
|
Note
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console. |
Dynamic time expressions are also supported.
Examples:
Basic example
"QueryParameters": {
"type": "post",
"expand": "true",
"id": 120
}
Dynamic time expressions
"QueryParameters": {
"since": "@Now-6M",
"until": "@Now"
}
Authentication with an API key as a query parameter
If your API requires Coveo to authenticate with an API key as the value of the api_key
query parameter, you must enter this key in the Authentication section of the Add a REST API source panel.
![API key in the source panel](/en/assets/images/index-content/source-ref-apikeyplaceholder.png)
Then, in your source JSON configuration, you must use the @ApiKey
placeholder to refer to this key.
This ensures that your API key isn’t stored in clear text in the source JSON configuration, where other Administration Console users could see it.
When Coveo processes the source configuration, it replaces the placeholder with the actual key you entered in the panel.
Your source configuration could therefore look like this.
{
"Services": [
{
"Url": "http://example.com/api/v1",
"Endpoints": [
{
"Path": "/users",
"Method": "GET",
"ItemType": "User",
"Uri": "%[coveo_url]/users/%[id]",
"ClickableUri": "%[coveo_url]/users/%[id]",
"QueryParameters": {
"api_key": "@ApiKey"
}
}
]
}
]
}
api_key is the name of the query parameter that your API expects to contain the API key. |
As a result, Coveo sends the following HTTP request to the API:
GET /users?api_key=Az4adfSyBhZX8fTThUJ4Kb2TC3Ax0RshgrZtUiqM HTTP/1.1
Host: http://example.com/api/v1
Az4adfSyBhZX8fTThUJ4Kb2TC3Ax0RshgrZtUiqM
is the API key you entered in the source panel.
RefreshEndpoints
(array)
Each object in the RefreshEndpoints
array is a refresh endpoint query to execute to retrieve added, updated, or deleted items when refresh is supported and performed.
A refresh endpoint is an endpoint that inherits from the parent endpoint definition.
The refresh endpoint allows you to redefine the endpoint configuration properties.
As a result, when you refresh your source, the properties defined under RefreshEndpoints
override the properties defined under Endpoints
.
When you rescan or rebuild the source, however, the Endpoints
properties apply.
Similarly, if you don’t specify any QueryParameters
in the refresh endpoint definition, the endpoint query parameters will be inherited.
However, either all query parameters are inherited, or none is.
So, if you change one parameter, you must also redefine all other parameters that apply, even if they’re identical to those in the endpoint object.
Your endpoint query parameters are the following:
"QueryParameters": {
"type": "post",
"workflow_state":"published",
"since": "@RefreshDate",
"expand": "true"
}
In your refresh endpoint configuration, you want to switch expand
to false
and keep all other query parameters identical, so you enter the following:
"QueryParameters": {
"type": "post",
"workflow_state":"published",
"since": "@RefreshDate",
"expand": "false"
}
|
Note
Dynamic values aren’t supported in the query parameters of a |
In addition to the Endpoints
object configuration properties, the RefreshEndpoints
object supports the following properties:
Example:
"RefreshEndpoints":[
{
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters":{
"workflow_state":"published",
"since": "@RefreshDate"
}
},
{
"IsDeletionQuery":true,
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters":{
"lastModifiedDate": "@Now-7d",
"status":"Deleted"
}
},
{
"IsDeletionQuery":true,
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"Path":"/api/table/audit_delete",
"QueryParameters":{
"tablename":"kb_knowledge",
"since": "@RefreshDate"
}
},
{
"IsDeletedItem": "%[is_deleted]",
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters":{
"since": "@RefreshDate"
}
}
]
IsDeletionQuery
(Boolean)
Services
> Endpoints
> RefreshEndpoints
> IsDeletionQuery
Whether the defined query returns items to delete from the index.
Default value is false
, which means that all retrieved items are added to the index or updated.
If the query returns items to add, update, and delete all at once, use the IsDeletedItem
property instead.
IsDeletedItem
(Boolean)
Services
> Endpoints
> RefreshEndpoints
> IsDeletedItem
When a refresh query returns items to add, update, and delete all at once, this property allows you to specify which of these items should be deleted.
Default value is false
, but it’s considered to be true
if IsDeletionQuery
is set to true
.
You can provide a dynamic value (for example, %[has_been_deleted_flag]
) to be evaluated for each item or a static value.
Example:
"%[is_deleted]"
deletes all items that have a "is_deleted": true
JSON property flagging them as deleted.
{
"IsDeletedItem": "%[is_deleted]",
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters":{
"since": "@RefreshDate"
}
}
DeleteChildren
(Boolean)
Services
> Endpoints
> RefreshEndpoints
> DeleteChildren
Whether to delete children when the retrieved parent item is deleted.
Default value is false
.
If the value is set to true
, child items are deleted based on their Uri
.
SubItems
(array)
A list of sub-items to retrieve.
Each sub-item is represented by a configuration object under Endpoints
.
An item can have many different sub-items, for example, a blog post containing comments and attachments as child items.
The SubItems
object establishes a parent-child relationship between items.
There’s no restriction on the number of sub-items an item can have.
In addition, sub-items can also have sub-items, for example if blog post comments also have attachments.
In the application JSON response, these sub-items are nested within each other, and so should be your SubItems
arrays in your source configuration.
To refer to the metadata of a parent item, prefix the metadata name with coveo_parent
.
You may also need to use raw
.
Examples:
When the sub-items to index are located under a different endpoint, you provide the Path
to their location.
"SubItems": [
{
"Path": "/posts/%[coveo_parent.id]/comments",
"Method": "GET",
"ItemType": "Comment",
"Uri": "%[coveo_url]/repository/posts/%[coveo_parent.id]/comments/%[id]",
"ClickableUri": "%[coveo_url]/posts/%[coveo_parent.id]/comments/%[id]",
"Title": "%[title]",
"ModifiedDate": "%[updated]",
"Body": "%[content]",
"Metadata": {
"author": "%[author]",
"createdby" :"%[coveo_parent.author]",
"id": "%[id]"
}
}
]
Alternatively, when the desired sub-items are located under the same endpoint as their parent items (for example, the child item data is nested within the parent item data), you omit the Path
property and provide the sub-items' location using the ItemPath
property.
"SubItems": [
{
"ItemPath": "/comments",
"ItemType": "Comment",
"Uri": "%[coveo_url]/repository/posts/%[coveo_parent.id]/comments/%[id]",
"ClickableUri": "%[coveo_url]/posts/%[coveo_parent.id]/comments/%[id]",
"Title": "%[title]",
"ModifiedDate": "%[updated]",
"Body": "%[content]",
"Metadata": {
"author": "%[author]",
"createdby" :"%[coveo_parent.author]",
"id": "%[id]"
}
}
]
SubQueries
(array)
Each object in the SubQueries
array represents a subquery to execute on every item in order to fetch additional metadata and/or the item body.
Subqueries are especially useful when this information is located under a different path than that of the item.
To refer to the metadata of a parent item, prefix the metadata name with coveo_parent
.
You may also need to use raw
.
Example:
"SubQueries": [
{
"Path": "/posts/%[coveo_parent.id]/additionalinfo",
"Method": "GET",
"Body": "%[data.body]",
"Metadata": {
"created": "%[created]"
}
}
]
The subquery configuration supports the following properties from the Endpoints
object configuration:
In addition, it supports the following property from the Services
object configuration:
It also contains the following property:
IsThumbnail
(Boolean)
Services
> Endpoints
> SubQueries
> IsThumbnail
If the SubQueries
object path leads to an image, set IsThumbnail
to true
to show this image as the thumbnail of an item.
The default value is false
.
Supported thumbnail file formats are the following:
.bmp
, .emf
, .exif
, .gif
, .icon
, .jpeg
, .png
, .tiff
, .wmf
.
|
Notes
|
Title
(string)
The title of the item. Although this property isn’t required in the JSON configuration, we recommend that you include it, as the Content Browser displays its value.
Typically, Title
contains a dynamic value, since a static, hardcoded value would make all item titles identical.
Example: "Title": "%[title]"
Url
(string, required)
Services
> Url
The Url
value is the service URL of your web application.
Example: "Url": "http://example.com/api/v1"
Authentication
(object)
Services
> Authentication
If your API service requires Coveo to authenticate, you must enter the necessary credentials in the Authentication section of the Add a REST API source panel.
Then, in your source JSON configuration, you must specify how Coveo should use these credentials to authenticate with the API.
If your API uses password-based or OAuth 2.0 authentication, you must use the Authentication
object, which contains the information Coveo will use to authenticate to the service.
What if my API uses API keys?
If your API uses API keys for authentication, Coveo will typically need to provide such a key in an HTTP header, as a query parameter, or as a payload parameter.
You therefore don’t need an Authentication
object in your source JSON configuration.
In the Authentication
object, you’ll use a placeholder starting with @
to refer to the credentials you entered in the panel.
This ensures that your credentials aren’t stored in clear text in the source JSON configuration, where other Administration Console users could see them.
Then, when Coveo processes the source configuration, it replaces the placeholders with the actual credentials you entered in the panel.
The following table shows the objects and placeholders you must include in your source configuration, depending on the type of authentication your API uses:
Authentication type | Use placeholders in | Placeholders to use |
---|---|---|
Basic, HTTP, Kerberos, or NTLM (password-based) |
|
|
OAuth 2.0 |
|
|
API key |
|
|
As indicated in the table above, API key placeholders don’t appear in the Authentication
object.
If your API authenticates with API keys, see the sections listed in the table for examples with API key placeholders.
Examples:
Basic, HTTP, Kerberos, or NTLM
If your API uses password-based authentication, you must enter the username and password in the Authentication section of the Add a REST API source panel.
![Username and password in the source panel](/en/assets/images/index-content/source-ref-usernamepasswordplaceholder.png)
Then, in your source JSON configuration, you must use the @Username
and @Password
placeholders to refer to these credentials.
Your Authentication
object therefore looks like this:
"Authentication": {
"Username": "@Username",
"Password": "@Password",
"ForceBasicAuthentication": "true"
}
OAuth 2.0
If your API uses OAuth 2.0 authentication, you must enter the client ID, client secret, and refresh token in the Authentication section of the Add a REST API source panel.
![OAuth information in the source panel](/en/assets/images/index-content/source-ref-oauthplaceholder.png)
Then, in your source JSON configuration, you must use the @ClientId
, @ClientSecret
, and @RefreshToken
placeholders to refer to this information.
Your Authentication
object could therefore look like this:
"Authentication": {
"OAuth": {
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
}
}
}
}
}
The Authentication
object supports the following properties:
-
OAuth
(required if authenticating with OAuth 2.0)
Username
(string)
Services
> Authentication
> Username
Username used to log in to the service.
You can use @Username
to retrieve the value specified in the Add a REST API Source panel.
Password
(string)
Services
> Authentication
> Password
Password used to log in to the service.
This property can be empty even though a Username
is specified.
You can use @Password
to retrieve the value specified in the Add a REST API Source panel.
Domain
(string)
Services
> Authentication
> Domain
Domain name to use when authenticating to a NTLM or Kerberos protected server.
ForceBasicAuthentication
(Boolean)
Services
> Authentication
> ForceBasicAuthentication
Whether to force a basic HTTP header in the request.
The default value is false
.
OAuth
(object)
Services
> Authentication
> OAuth
If your application requires OAuth 2.0 authentication, add an OAuth
object to your source configuration.
Your OAuth
object must contain a Query
object.
Coveo will use the information in the Query
object to make an authentication query to your application.
Optionally, your OAuth
object may also contain a Response
object.
In this object, you may provide parameters such as AccessToken
, TokenType
, and ExpiresIn
, whose values indicate where Coveo will find the corresponding information when parsing the API’s response.
The Response
object may also include the AuthorizationHeader
parameter, whose value is the name of the authorization header that Coveo should use when querying your API for content.
|
Enter your client ID, client secret, and refresh token in the Add a REST API source panel and use the This allows you to keep your confidential information obfuscated, thus preventing other Administration Console users to access it. ![]() |
Example:
"OAuth": {
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
},
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
}
The OAuth
object supports the following objects:
Query
(object, required)
Services
> Authentication
> OAuth
> Query
If your application requires OAuth 2.0 authentication, you must add an OAuth
object to your source configuration.
Your OAuth
object must contain a Query
object.
Coveo will use the information in the Query
object to make an authentication query to your application.
|
Enter your client ID, client secret, and refresh token in the Add a REST API source panel and use the This allows you to keep your confidential information obfuscated, thus preventing other Administration Console users to access it. ![]() |
Example:
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}
The Query
object supports the following properties:
-
RefreshUrl
(required)
RefreshUrl
(string, required)
Services
> Authentication
> OAuth
> Query
> RefreshUrl
If your application requires OAuth 2.0 authentication, your Query
object must include a RefreshUrl
property.
The RefreshUrl
value must be a URL that Coveo will use to refresh its access token.
Example
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}
OAuth query Headers
(object)
Services
> Authentication
> OAuth
> Query
> Headers
If your application requires OAuth 2.0 authentication, your Query
object can optionally include a Headers
object.
When making an authentication query to your application, Coveo will add these HTTP headers to its request.
|
Note
Coveo automatically sets the Content-Type header.
Therefore, you don’t need to add it to the |
Example:
"Query": {
"RefreshUrl": "http://example.com/token",
"Headers": {
"X-Auth": "apikey1234567890abcdef"
}
}
OAuth query Method
(string enum)
Services
> Authentication
> OAuth
> Query
> Method
If your application requires OAuth 2.0 authentication, your Query
object can optionally include a Method
property.
The Method
property specifies the HTTP method that Coveo should use when refreshing its token.
Default is POST
.
Example
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}
OAuth query Parameters
(object)
Services
> Authentication
> OAuth
> Query
> Parameters
If your application requires OAuth 2.0 authentication, your Query
object can optionally include query parameters for Coveo to use.
Each object in the Parameters
object represents a parameter to include in Coveo’s authentication request to your application.
Each object should contain a "Type" and a "Value" property.
The Type
property indicates how to include the parameter in the request.
Possible values for this property are query
or payload
.
In addition, if you include a refresh_token
parameter object, you should also add a IsRefreshToken
property with a value of true
to this object.
|
Enter your client ID, client secret, and refresh token in the Add a REST API source panel and use the This allows you to keep your confidential information obfuscated, thus preventing other Administration Console users to access it. ![]() |
Example:
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}
Response
(object)
Services
> Authentication
> OAuth
> Response
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
The Response
object contains, among other things, the properties that Coveo must extract from the authentication server’s response.
The value of each property represents the path to the corresponding value in the response.
In the Response
object, you may also enter other authentication parameters, such as the AuthorizationHeader
and SupportsRefreshToken
properties.
Example:
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json
{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}
Then your Response
object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
The Response
object supports the following properties:
AccessToken
(string)
Services
> Authentication
> OAuth
> Response
> AccessToken
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
Coveo will then extract the information from the API’s response using the values specified in the Response
object.
In the Response
object, you can add the AccessToken
property to specify the path to the access token in the API’s response.
Default is access_token
.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json
{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}
Then your Response
object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
SupportsRefreshToken
(Boolean)
Services
> Authentication
> OAuth
> Response
> SupportsRefreshToken
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
In the Response
object, you can add the SupportsRefreshToken
property to indicate whether your application’s OAuth 2.0 flow supports refresh tokens.
Default is true
.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json
{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}
Then your Response
object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
RefreshToken
(string)
Services
> Authentication
> OAuth
> Response
> RefreshToken
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
Coveo will then extract the information from the API’s response using the values specified in the Response
object.
In the Response
object, you can add the AccessToken
property to specify the path to the refresh token in the API’s response.
Default is refresh_token
.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json
{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}
Then your Response
object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
ExpiresIn
(string)
Services
> Authentication
> OAuth
> Response
> ExpiresIn
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
Coveo will then extract the information from the API’s response using the values specified in the Response
object.
In the Response
object, you can add the ExpiresIn
property to specify the path to the refresh token expiration delay in the API’s response.
Default is expires_in
.
If the API’s response doesn’t contain an expiration delay, use the ExpiresInDefaultValue
property instead.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json
{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}
Then your Response
object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
ExpiresInDefaultValue
(number)
Services
> Authentication
> OAuth
> Response
> ExpiresInDefaultValue
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
Coveo will then extract the information from the API’s response using the values specified in the Response
object.
Typically, the authentication server will return an expiration delay for its refresh token, which you can extract with the ExpiresIn
property.
However, if the API’s response doesn’t include an expiration delay, you can add the ExpiresInDefaultValue
property to your Response
object.
The ExpiresInDefaultValue
value specifies the number of seconds after which Coveo should consider the refresh token expired.
Default is 3600
.
Example:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresInDefaultValue": 1800,
"TokenType": "token_type"
}
TokenType
(string)
Services
> Authentication
> OAuth
> Response
> TokenType
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
Coveo will then extract the information from the API’s response using the values specified in the Response
object.
In the Response
object, you can add the ExpiresIn
property to specify the path to the token type in the API’s response.
Default is token_type
.
If the response doesn’t specify a token type or if it’s invalid or inappropriate, Coveo will use Bearer
by default.
If you want a different default value, use the TokenTypeDefaultValue
property instead.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json
{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}
Then your Response
object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
TokenTypeDefaultValue
(string)
Services
> Authentication
> OAuth
> Response
> TokenTypeDefaultValue
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
Coveo will then extract the information from the API’s response using the values specified in the Response
object.
Typically, the authentication server will return a token type, which you can extract with the TokenType
property.
If the API’s response doesn’t specify a token type, or if the token type is invalid or inappropriate, Coveo will use Bearer
by default.
However, if you want Coveo to use a different default token type, you can specify it by the TokenTypeDefaultValue
property to your Response
object.
AuthorizationHeader
(string)
Services
> Authentication
> OAuth
> Response
> AuthorizationHeader
If your application requires OAuth 2.0 authentication, you can optionally add a Response
object to your OAuth
object.
In the Response
object, you can add the AuthorizationHeader
property to indicate the name of the authorization header that Coveo should use to provide its access token when querying your API for content.
Refer to your API’s documentation for the header name to use.
Default is Authorization
.
|
Notes
|
Example:
If your API documentation specifies that the access token must be provided in a header named X-ACCESS-TOKEN
, then your Response
object should specify it like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type",
"AuthorizationHeader": "X-ACCESS-TOKEN"
}
As a result, when making content requests to your application, Coveo will automatically include an X-ACCESS-TOKEN
header with the access token value.
{
"headers": {
"Content-Type": "application/json",
"X-ACCESS-TOKEN": "123456789abcdef"
}
}
Headers
(object)
Services
> Headers
A key-value list of HTTP headers to add to the query. Each value can be either a number, string, or Boolean.
The Headers
object is inheritable.
|
Note
Unlike a browser or a tool such as Postman, Coveo doesn’t automatically add a |
Examples:
Basic example
"Headers": {
"Custom-Header-Name": "myValue"
}
Authentication with an API key in the Authorization
HTTP header
If your API requires Coveo to authenticate with an API key in the Authorization
HTTP header, you must enter this key in the Authentication section of the Add REST API source panel.
![API key in the source panel](/en/assets/images/index-content/source-ref-apikeyplaceholder.png)
Then, in your source JSON configuration, you must use the @ApiKey
placeholder to refer to this key.
This ensures that your API key isn’t stored in clear text in the source JSON configuration, where other Administration Console users could see it.
When Coveo processes the source configuration, it replaces the placeholder with the actual key you entered in the panel.
Your source configuration could therefore look like this.
{
"Services": [
{
"Url": "http://example.com/api/v1",
"Headers": {
"Authorization": "Bearer @ApiKey"
},
"Endpoints": [
{
"Path": "/users",
"Method": "GET",
"ItemType": "User",
"Uri": "%[coveo_url]/users/%[id]",
"ClickableUri": "%[coveo_url]/users/%[id]"
}
]
}
]
}
Bearer is the authorization type and is most commonly used with OAuth 2.0 tokens.
If your API requires a different authorization type, replace Bearer with the appropriate type. |
As a result, Coveo sends the following HTTP request to the API:
GET /users HTTP/1.1
Host: http://example.com/api/v1
Authorization: Bearer Az4adfSyBhZX8fTThUJ4Kb2TC3Ax0RshgrZtUiqM ![1](/en/assets/icons/callouts/1.svg)
Az4adfSyBhZX8fTThUJ4Kb2TC3Ax0RshgrZtUiqM is the API key you entered in the source panel. |
Authentication with an API key in a custom HTTP header
If your API requires Coveo to authenticate with an API key in the custom HTTP header X-Api-Key
, you must enter this key in the Authentication section of the Add REST API source panel.
![API key in the source panel](/en/assets/images/index-content/source-ref-apikeyplaceholder.png)
Then, in your source JSON configuration, you use the @ApiKey
placeholder to refer to this key.
This ensures that your API key isn’t stored in clear text in the source JSON configuration, where other Administration Console users could see it.
When Coveo processes the source configuration, it replaces the placeholder with the actual key you entered in the panel.
Your source configuration could therefore look like this.
{
"Services": [
{
"Url": "http://example.com/api/v1",
"Headers": {
"X-Api-Key": "@ApiKey"
},
"Endpoints": [
{
"Path": "/users",
"Method": "GET",
"ItemType": "User",
"Uri": "%[coveo_url]/users/%[id]",
"ClickableUri": "%[coveo_url]/users/%[id]"
}
]
}
]
}
X-Api-Key is the name of the custom header that your API expects to contain the API key. |
As a result, Coveo sends the following HTTP request to the API:
GET /users HTTP/1.1
Host: http://example.com/api/v1
X-Api-Key: Az4adfSyBhZX8fTThUJ4Kb2TC3Ax0RshgrZtUiqM ![1](/en/assets/icons/callouts/1.svg)
Az4adfSyBhZX8fTThUJ4Kb2TC3Ax0RshgrZtUiqM is the API key you entered in the source panel. |
Paging
(object)
Services
> Paging
In the Paging
object, specify how you want the content to be paged.
This property is inheritable, but you can disable inheritance using the DoNotInherit
property.
When you set it at the service level, it will apply to all endpoints in this service by default.
You can override the service-level configuration by setting the Paging
object again at the endpoint level.
Sub-items inherit this property.
Sub-queries don’t inherit this property.
Example:
"Paging": {
"PageSize": 20,
"OffsetStart": 0,
"OffsetType": "page",
"Parameters": {
"Limit": "limit",
"Offset": "page"
},
"DoNotInherit": true
}
The Paging
object supports the following properties:
-
OffsetType
(required) -
PageSize
(required)
OffsetType
(string enum, required)
When configuring your source to request paginated content, you must specify how your API paginates its content. Coveo needs this information to build its page request URLs in the appropriate format. You can typically find it in your API’s documentation.
The OffsetType
property is a string enum that specifies the type of pagination your API uses.
The value to specify depends on your API’s pagination technique, just like the parameters you’ll need to add to your Paging
object.
Pagination technique |
OffsetType value |
Parameters to add to the Paging object |
---|---|---|
Page-based |
|
|
Offset-based |
|
|
URL-based |
|
|
Cursor-based |
|
Examples:
Page-based pagination
Your API expects request URLs in a format such as http://example.com/api/item?limit=100&pageNumber=0
for the first page, and http://example.com/api/item?limit=100&pageNumber=1
for the second page.
With these URLs, Coveo would request the first page (page 0), and then increment the page number by 1 to request the second page, and so on.
In this case, your Paging
object should look like this:
{
"Paging": {
"PageSize": 100,
"OffsetStart": 0,
"OffsetType": "page",
"Parameters": {
"Limit": "limit",
"Offset": "pageNumber"
}
}
}
Offset-based pagination
Your API expects request URLs in a format such as http://example.com/api/item?limit=100&start=0
for the first page, and http://example.com/api/item?limit=100&start=100
for the second page.
With these URLs, Coveo would request the 100 first items starting from item 0, and then the 100 next items, starting from item 100.
In this case, your Paging
object should look like this:
{
"Paging": {
"PageSize": 100,
"OffsetStart": 0,
"OffsetType": "item",
"Parameters": {
"Limit": "limit",
"Offset": "start"
}
}
}
URL-based pagination
Your API expects request URLs in a format such as http://example.com/api/item?limit=8
for the first page, and http://example.com/api/item?page=3d170d80d8n3n2342c328s
for the second page.
In this context, Coveo would request the first page, and then the next page using the URL provided in the nextPage
value in the JSON response.
In this case, if your API returns the following first page:
{
"nextPage": "http://example.com/api/item?page=3d170d80d8n3n2342c328s",
"items": [
{ "id": 1, "name": "Caroline" },
{ "id": 2, "name": "Marcella" },
{ "id": 3, "name": "Susie" },
{ "id": 4, "name": "Rhonda" },
{ "id": 5, "name": "Wendy" },
{ "id": 6, "name": "Barbara" },
{ "id": 7, "name": "Deirdre" },
{ "id": 8, "name": "Lynda" }
]
}
Then your Paging
object should look like this:
{
"Paging": {
"PageSize": 8,
"NextPageKey": "nextPage",
"OffsetType": "url",
"Parameters": {
"Limit": "limit"
}
}
}
Cursor-based pagination
Your API expects request URLs in a format such as http://example.com/api/item?limit=8
for the first page, and http://example.com/api/item?limit=8&nextPageToken=3d170d80-7b3b-4377
for the second page.
With these URLs, Coveo would request the 8 first items, and then the next page using the token provided in the NextPageId
value in the JSON response.
In this case, if your API returns the following first page:
{
"nextPageId": "3d170d80-7b3b-4377",
"items": [
{ "id": 1, "name": "Caroline" },
{ "id": 2, "name": "Marcella" },
{ "id": 3, "name": "Susie" },
{ "id": 4, "name": "Rhonda" },
{ "id": 5, "name": "Wendy" },
{ "id": 6, "name": "Barbara" },
{ "id": 7, "name": "Deirdre" },
{ "id": 8, "name": "Lynda" }
]
}
Then your Paging
object should look like this:
{
"Paging": {
"PageSize": 8,
"NextPageKey": "NextPageId",
"OffsetType": "cursor",
"Parameters": {
"Limit": "limit",
"Offset": "nextPageToken"
}
}
}
PageSize
(number, required)
The number of items to fetch per page.
Example: With paging URL https://example.com/api/item?position=0&quantity=50
, your JSON configuration should include "PageSize": 50
.
DoNotInherit
(Boolean)
Whether to prevent your Paging
configuration from being inherited.
This can be useful for preventing paging on PermissionSubqueries
, which is known to slow down the crawling process.
The default value is false
.
NextPageKey
(string)
When the OffsetType
value is "url"
or "cursor"
, provide the path (simple path or JSONPath) to the key representing the value of the next page reference, that is, either the next page URL or the next page cursor.
The NextPageKey
property isn’t required when the OffsetType
value is page
or item
.
Example: "NextPageKey": "NextPageLink"
OffsetStart
(number)
Offset of the first page to fetch.
This property is required when the OffsetType
value is "page"
or "item"
.
Example: Your first paging URL is https://example.com/api/item?position=0&quantity=50
.
If you want to retrieve all items except items 1 to 50, your paging configuration must include "OffsetStart": 1
.
Paging Parameters
(object)
In the paging Parameters
object, you must provide the names of the query string fields in your web application pagination URL that are required to support pagination.
The parameters expected in this object are Limit
and Offset
.
However, the Offset
parameter isn’t required when the OffsetType
value is "url"
since the complete URL is already retrieved in the JSON response.
Example:
"Parameters": {
"Limit": "limit",
"Offset": "page"
}
The Parameters
object supports the following properties:
-
Limit
(required)
Limit
(string, required)
Services
> Paging
> Parameters
> Limit
In the query string, the field whose value is retrieved with the PageSize
parameter.
Example: With paging URL https://example.com/api/item?position=0&quantity=50
, your JSON configuration should include "Limit": "quantity"
.
FetchNextPageUntilNoResult
(Boolean)
Services
> Paging
> Parameters
> FetchNextPageUntilNoResult
The FetchNextPageUntilNoResult
property indicates whether Coveo will try to fetch pages until no result is returned, or until the application API returns an error.
Default value is true
.
The FetchNextPageUntilNoResult
property is an alternative to the TotalCountHeaderKey
and TotalCountKey
properties.
|
Unless you set this property to |
Example:
"Paging": {
"OffsetType": "page",
"PageSize": 25,
"Parameters": {
"FetchNextPageUntilNoResult": true
}
}
Offset
(string)
Services
> Paging
> Parameters
> Offset
In the query string, the field whose value is retrieved with the OffsetStart
parameter.
Example: Your first paging URL is https://example.com/api/item?position=0&quantity=50
.
If you want to ignore items 1 to 50 and retrieve items 51 and over, your paging configuration must include "Offset": "position"
and "OffsetStart": 1
.
TotalCountHeaderKey
(string)
The name of an HTTP header whose value represents the total number of items retrieved through the API call. This property is useful when Coveo makes an API call to a page that doesn’t exist, causing an error to be returned.
However, when using this property, you must be sure that the total always matches the actual number of items returned by your API, as Coveo’s crawler uses this total as an indication to stop. Should the expected total be lower than the actual number of items returned, Coveo won’t index the remaining items.
Therefore, if your API supports it, we recommend using the FetchNextPageUntilNoResult
property instead, as a safer alternative.
On the other hand, if the expected total is higher than the actual number of items returned, Coveo’s crawler will throw an error after 10 empty pages.
In such a case, you can download the activity logs to review the TotalCountKey
value that Coveo retrieved, and then adjust your source configuration accordingly.
Example: "TotalCountHeaderKey": "x-total-count"
TotalCountKey
(string)
The path (simple path or JSONPath) to a response body property representing the total number of items retrieved through the API call. This property is useful when Coveo makes an API call to a page that doesn’t exist, causing an error to be returned.
However, when using this property, you must be sure that the total always matches the actual number of items returned by your API, as Coveo’s crawler uses this total as an indication to stop. Should the expected total be lower than the actual number of items returned, Coveo won’t index the remaining items.
Therefore, if your API supports it, we recommend using the FetchNextPageUntilNoResult
property instead, as a safer alternative.
On the other hand, if the expected total is higher than the actual number of items returned, Coveo’s crawler will throw an error after 10 empty pages.
In such a case, you can download the activity logs to review the TotalCountKey
value that Coveo retrieved, and then adjust your source configuration accordingly.
Example: "TotalCountKey": "retrievedContent.totalCount"
Permissions
(array)
Services
> Permissions
Each object in the Permissions
array represents a permission level.
In turn, permission levels contain one or more PermissionsSets
arrays.
The Permissions
array is required when you select the Same users and groups as in your content system source content security option.
It contains the permission levels and permission sets to apply to the items indexed through this service.
The members listed in these levels and sets are either allowed or denied access to the indexed items.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
This property is inheritable.
Example:
"Permissions": [
{
"Name": "Permission Level 1",
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "jsmith@example.com",
"Type": "User",
"AdditionalInfo": {
"title": "%[customMetadata]"
}
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": false
}
]
}
]
The Permissions
array supports the following properties:
PermissionsSets
(array)
Services
> Permissions
> PermissionsSets
The PermissionsSets
array specifies who can or can’t access the indexed items.
Each object in the array describes a single permission set, which can contain lists of allowed and denied members.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
Objects in the PermissionsSets
array support the following properties:
Permission set Name
(string)
Services
> Permissions
> PermissionsSets
> Name
The Name
property indicates the name of the permission set.
AllowedMembers
and DeniedMembers
arrays
Services
> Permissions
> PermissionsSets
> AllowedMembers
/DeniedMembers
Each object in the AllowedMembers
and DeniedMembers
arrays represents a member security identity that should be allowed or denied to access the indexed content.
|
Note
If a user is listed under both |
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
The following is an example of a simple situation where, in your permission system, security identities are identified with email addresses. Your source will work with the Email security identity provider to ensure that your search interface users only see the content they’re allowed to.
If your permission system is more complex, requires a security provider other than Email, or is frequently updated, you should not only index allowed and denied security identities, but also provide a permission configuration instructing Coveo on how to index your permission system.
You index text items written by different people.
The metadata of each item contains the key author
, and, in this system, authors are identified by their email address.
These addresses are therefore expected as values of the Name
field.
With the following configuration, each user in your organization can access the items they have created through your Coveo-powered search interfaces.
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "%[author]",
"Type": "User",
"Condition": "%[published] == true",
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": false
}
]
AllowedMembers
and DeniedMembers
arrays support the following properties:
-
Name
(required)
Member Name
(string, required)
Services
> Permissions
> PermissionsSets
> AllowedMembers
/DeniedMembers
> Name
The name of the member security identity.
In simple use cases similar to that described under AllowedMembers
and DeniedMembers
arrays, this should be an email address.
Dynamic values are supported.
Member Type
(string enum, required)
Services
> Permissions
> PermissionsSets
> AllowedMembers
/DeniedMembers
> Type
The type of member.
Allowed values are User
, Group
, and VirtualGroup
.
For more information on these types of security identities, see Coveo management of security identities and item permissions.
Permission Condition
(string)
Services
> Permissions
> PermissionsSets
> AllowedMembers
/DeniedMembers
> Condition
The Condition
value is a condition that must resolve to true for the security identity specified in Name
to be resolved, that is, extracted by the security identity provider.
As a result, the permission applies.
When the condition resolves to false, the security identity is not resolved and the corresponding permission doesn’t apply.
Adding a condition to your AllowedMembers
or DeniedMembers
object is optional.
To write your condition, use the same syntax as for indexing pipeline extension conditions.
Conditions can be assembled using the following operators: AND
, OR
, Exists
, NOT
, >
, and <
.
Parentheses are also supported to specify operation order.
The metadata fields in your condition must either be defined in the Metadata
object of the source configuration or referenced with raw
.
You can specify an array if the metadata refers to a multi-value field in your repository.
Dynamic values are also supported.
You index a list of available and discontinued products, and your public search interface is used by both your employees and your customers. On this search interface, your employees can log in to see additional content, while customers can’t log in.
You want to hide your discontinued products from your customers only, so you use the Condition
parameter to apply a permission to these items.
As a result, all products with Discontinued: yes
in their metadata are visible to members of the *@mycompany.com
group only, that is, your employees, while non-discontinued products are available to all search interface users.
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "*@mycompany.com",
"Type": "Group",
"Condition": "%[discontinued] == yes"
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": true
}
]
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
PermissionType
(string)
Services
> Permissions
> PermissionsSets
> AllowedMembers
/DeniedMembers
> PermissionType
This parameter is relevant only when you provide a permission configuration.
The PermissionType
is a key representing the configuration on how to extract all relationships of the security identity specified in Name
.
The security identity provider will use this configuration when processing the security identity.
In the Content Security tab of the Add/Edit a REST API Source panel, you’ll need to select the Same users and groups as in your content system, and then to provide a JSON permission configuration detailing how to retrieve the relationships of each security identity and how to index this data.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
AdditionalInfo
(object)
Services
> Permissions
> PermissionsSets
> AllowedMembers
/DeniedMembers
> AdditionalInfo
This parameter is relevant only when you provide a permission configuration.
The AdditionalInfo
object is a key-value store of additional information required by some security providers.
Each key represents the metadata name of the repository item, while its value is the value path (simple path or JSONPath) in the JSON response.
When writing your permission configuration, you can use coveo_parent
to refer to the information retrieved with the AdditionalInfo
parameter of a permission subquery.
Dynamic values are supported in the values of this object.
IsAnonymousAllowed
(Boolean)
Permissions
> PermissionsSets
> IsAnonymousAllowed
The IsAnonymousAllowed
property indicates whether anonymous, that is, unauthenticated users are allowed to access the retrieved items.
Default value is false
.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
PermissionsFromMetadata
array
Services
> Permissions
> PermissionsSets
> PermissionsFromMetadata
This array is relevant only when you provide a permission configuration.
The PermissionsFromMetadata
array is an alternative to the PermissionSubQueries
array, as it fetches permission data, typically the security identities that are allowed or denied access to the item.
This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo.
Include a PermissionsFromMetadata
array in your source configuration when the permission data appears in the metadata of an item.
Conversely, a PermissionSubQueries
array should be used when an item and its permission data must be retrieved with two different queries.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
Objects of the PermissionsFromMetadata
array support:
PermissionSubQueries
array
Services
> Permissions
> PermissionsSets
> PermissionSubQueries
This array is relevant only when you provide a permission configuration.
Each object in the PermissionSubQueries
array represents a subquery to execute on every item to fetch permission data, typically the security identities that are allowed or denied access to the item.
This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo.
The PermissionSubQueries
array is an alternative to the PermissionsFromMetadata
array.
Include a PermissionSubQueries
array in your source configuration when an item and its permission data must be retrieved with two different queries. Conversely, the PermissionsFromMetadata
array should be used when the permission data appears in the metadata of an item.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
Objects of the PermissionSubQueries
array support:
-
The following properties from the
Endpoints
object configuration: -
The following properties from objects of the
AllowedMembers
andDeniedMembers
arrays: -
The following properties:
Example:
"permissionSubQueries": [
{
"ItemPath": "permissions",
"Path": "%[productid]/permissions",
"Method": "GET",
"QueryParameters": {},
"Name": "%[name]",
"Type": "group",
"PermissionType": "config1",
"Condition": "%[field] == value",
"IsAllowedMember": true,
"AdditionalInfo":
{
"PermissionID": "%[id]",
"PermissionName": "%[name]"
}
}
]
IsAllowedMember
(Boolean)
Services
> Permissions
> PermissionsSets
> PermissionSubQueries
> IsAllowedMember
The IsAllowedMember
property indicates whether the specified member is allowed to access the item.
Default value is false
.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
StringItemPath
(string)
Services
> Permissions
> PermissionsSets
> StringItemPath
The path to the item in the JSON response returned by your API.
When indexing permissions with PermissionSubQueries, use StringItemPath
in place of ItemPath
if your API returns the list of identities in an array rather than as a series of objects.
Then, use value %[item]
to tell Coveo where you want to store each of the identities extracted from the permission array.
The Name
property is an appropriate place.
Read more on StringItemPath vs. ItemPath
Most APIs return permissions as a series of objects such as:
{
"allowed": [
{
"email": "jsmith@example.com"
},
{
"email": "ballen@example.com"
}
]
}
In such a case, you would use ItemPath
to extract permissions.
However, if your API rather returns permissions as follows, you should use StringItemPath
instead.
{
"allowed": ["jsmith@example.com", "ballen@example.com"]
}
If you expect your API to return permissions in an array such as:
{
"allowedusers": ["jsmith@example.com", "ballen@example.com"]
}
Include the following in your permission subquery: "StringItemPath": "allowedusers"
.
In addition, to index the extracted user identities under Name
, add: "Name": %[item]
.
Your permission subquery therefore looks as follows:
"permissionSubQueries": [
{
"StringItemPath": "allowedusers",
"Path": "item/location",
"Method": "GET",
"Name": "%[item]",
"Type": "User",
"IsAllowedMember": true
}
]
StringItemPath
can appear in PermissionSubQueries objects only.
|
Note
|
SkippableErrorCodes
(string)
Services
> SkippableErrorCodes
When crawling your content, Coveo may encounter an HTTP error and stop.
If you expect the web application API to return an error and want it to be ignored so that the crawling process continues, specify the error code as the SkippableErrorCodes
property value.
This property is inheritable.
When you set it at the service level, it will apply to all endpoints in this service, and subqueries in these endpoints.
You can override the service-level configuration by setting the SkippableErrorCodes
property again at the endpoint or subquery level.
Sub-items don’t inherit this property.
Use a semicolon (;
) to separate error codes.
Example: "SkippableErrorCodes": "500;404;403"