SAP source reference

This is for:

Developer System Administrator

When creating an SAP source in the Coveo Administration Console, you must provide a JSON configuration detailing which services and endpoints to crawl to retrieve the desired content, and how to retrieve items of each type. This configuration has a single property, Services, whose value must be an array of Service Object.

Example
{
  "services": [
    {
      "url": "https://api.acmestore.com/odata2webservices",
      "headers": {
        "Accept": "application/json",
        "Application-Interface-Key": "xxxxxxxx"
      },
      "paging": {
        "pageSize": 10,
        "offsetStart": 0,
        "offsetType": "item",
        "nextPageKey": "__next",
        "parameters": {
          "limit": "$top",
          "offset": "$skip"
        }
      },
      "endpoints": [
        {
          "path": "/Product/Products",
          "method": "GET",
          "queryParameters": {
            "$filter": "catalogVersion/integrationKey eq 'Online%7CpowertoolsProductCatalog' and approvalStatus/code eq 'approved'",
            "$expand": "supercategories,thumbnails,picture,thumbnail,unit,catalogVersion/catalog,europe1Prices,europe1Prices/currency,bundleTemplates,localizedAttributes,supercategories/localizedAttributes,approvalStatus"
          },
          "itemPath": "d.results",
          "itemType": "product",
          "uri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]",
          "clickableUri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]",
          "metadata": {
            "objecttype": "Product",
            "ec_brand": "%[brand]",
            "ec_name": "%[name]",
            "name": "%[name]",
            "ec_item_group_id": "%[code]",
            "ec_shortdesc": "%[summary]",
            "ec_description": "%[description]",
            "ec_price": "%[europe1Prices.results[0].price]",
            "ec_thumbnails": "https://accstorefront.commerce.ondemand.com%[thumbnail.URL]",
            "ec_in_stock": "%[stock]",
            "ec_images": "https://accstorefront.commerce.ondemand.com%[picture.URL]",
            "ec_category": "%[supercategories.results[:].name]",
            "ec_pricerange": "%[priceRange]",
            "ec_configurable": "%[configurable]",
            "ec_code": "%[code]",
            "ec_url": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
          }
        }
      ]
    }
  ]
}

This reference article defines parameters to include in your JSON configuration.

When working on your SAP source, you may also want to refer to the following articles:

Tip

Use online tools such as JSONPath Online Evaluator to test your JSON paths, and freeformatter.com to validate and escape your JSON configuration.

Table of contents

To help you jump to the right section, here’s a hierarchical and alphabetical list of all supported arrays, objects, and parameters.

Show table of contents
Hierarchical list Alphabetical list

Services (required)

AccessToken

ActionOnItem

AdditionalInfo

AllowedMembers

Authentication

AuthorizationHeader

Body

ClickableUri

Condition (action on item)

Condition (permission)

DateFormat

DeleteChildren

DeniedMembers

Domain

DoNotInherit

Endpoints

Expand

ExpiresIn

ExpiresInDefaultValue

FetchNextPageUntilNoResult

Filter

ForceBasicAuthentication

Headers (OAuth query)

Headers (service level)

IndexingAction

IsAllowedMember

IsAnonymousAllowed

IsBinaryBody

IsDeletedItem

IsDeletionQuery

IsThumbnail

ItemPath

ItemType

Limit

Metadata

Method (endpoint query)

Method (OAuth query)

ModifiedDate

Name (member)

Name (permission level)

Name (permission set)

NextPageKey

OAuth

Offset

OffsetStart

OffsetType

PageSize

Paging

Parameters (OAuth query)

Parameters (paging)

Password

Path

PayloadJsonContent

PayloadParameters

PermanentId

Permissions

PermissionsFromMetadata

PermissionsSets

PermissionSubQueries

PermissionType

ProcessingAction

Query

QueryParameters

RefreshEndpoints

RefreshToken

RefreshUrl

Response

Services

SkippableErrorCodes

SubItems

SubQueries

SupportsRefreshToken

Title

TokenType

TokenTypeDefaultValue

TotalCountHeaderKey

TotalCountKey

Type (member)

Uri

Url

Username

UsingSingleUseRefreshToken

Services (array, required)

Each object in the Services array specifies an SAP API service to retrieve content, along with the specific endpoints to call.

The properties that may appear in an object of the Services array are the following:

Endpoints (array, required)

Services > Endpoints

Each object in the Endpoints array details how you want Coveo to crawl an endpoint of your SAP Commerce instance.

Moreover, in each endpoint object, the Metadata objects lists the content metadata to retrieve.

By default, the SAP source JSON configuration contains three endpoints, one for:

  • product (for example, "path": "/Product/Products")

  • variant (for example, "path": "FootWearVariant/GenericVariantProducts")

  • availability (for example, "path": "/Availability/Warehouses")

You can remove or add endpoints depending on your use case.

Objects of the Endpoints array support the following properties from their parent service object:

It also supports the following properties:

ClickableUri (string, required)

Services > Endpoints > ClickableUri

The ClickableUri is the URI of the page to open when a user clicks the title of an item from your SAP source in a Coveo-powered search interface. It can be identical to the Uri.

Dynamic values are supported.

Example: "ClickableUri": "%[coveo_url]/posts/%[id]"

ItemType (string, required)

Services > Endpoints > ItemType

The type of item that the resource contains. Coveo uses this data to organize the indexed content.

Space characters and dashes aren’t supported.

Example: "itemType": "product"

Path (string, required)

Services > Endpoints > Path

The relative path to the desired resource endpoint.

It’s made in the format of /name of the integration object created in the Backoffice/the type of the object+s.

Dynamic values are supported when specified within the SubItems, SubQueries, and PermissionSubQueries arrays only.

Dynamic time expressions are also supported.

Example: "path": "/Product/Products"

In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.

Uri (string, required)

Services > Endpoints > Uri

The URI of the product’s PDP page.

Dynamic values are supported.

Example: "Uri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"

Should a dynamic value be invalid or unretrievable, the source stops crawling and displays an error in the Administration Console.

Body (string)

Services > Endpoints > Body

The path (simple path or JSONPath) to the Body content of the item in the JSON response. The item body is displayed in the Content Browser.

Typically, the Body property contains a dynamic value, since a static, hardcoded value would make all item bodies identical. Dynamic values and hardcoded text can also be mixed.

Example: "Body": "%[html.content]"

DateFormat (string)

Services > Endpoints > DateFormat

The format of a date injected thanks to the tokens @Now and @RefreshDate.

All .NET formats are supported, and the default is yyyy-MM-ddTHH:mm:ssZ (ISO 8601).

Alternatively, to handle Unix timestamps, enter UnixEpoch.

Coveo also uses this format when parsing dates coming from your API. If these dates aren’t in a .NET format, use DateFormat to specify the format to expect.

Examples:

  • "DateFormat": "yyyy-MM-ddTHH:mm:ssZ"

  • "DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'"

IndexingAction (object)

Services > Endpoints > IndexingAction

Whether to ignore or to retrieve an item when a condition is met.

When the Condition resolves to true, the Coveo crawler applies the specified action to the item. Possible actions are Retrieve and Ignore, that is, the crawler can either index the item or ignore it.

An item ignored with IndexingAction isn’t indexed and therefore not visible in the Content Browser or a search interface. However, its sub-items are sent to the index, unless otherwise specified in the ProcessingAction object.

If the condition resolves to false, the Coveo crawler falls back to the default action, which is Retrieve, and the specified item is indexed.

Example:

With the following IndexingAction object, all items with an id greater than 5 are ignored. Only items with an id of 5 or less are indexed.

"IndexingAction": {
    "ActionOnItem": "Ignore",
    "Condition": "%[id] > 5"
}

The IndexingAction object supports the following properties:

ActionOnItem (string, required)

Services > Endpoints > IndexingAction > ActionOnItem

Action to perform if the specified Condition resolves to true.

In the IndexingAction object, this applies to retrieved items. Possible actions are Retrieve and Ignore, that is, the crawler can either index the item to make it searchable or ignore it.

In the ProcessingAction object, this applies to child items of an item. Possible actions are Process and Ignore, that is, the crawler can either retrieve the subitems to make them searchable or ignore them.

Action Condition (string)

Services > Endpoints > IndexingAction > Condition

The Condition value is a condition that must resolve to true for the specified action to apply to the child items of an item.

To write your condition, use the same syntax as for indexing pipeline extension conditions.

Conditions can be assembled using the following operators: AND, OR, Exists, NOT, >, and <.

Parentheses are also supported to specify operation order.

The metadata fields in your condition must either be defined in the Metadata object or referenced with raw. You can specify an array if the metadata refers to a multi-value field in your repository.

Examples:

  • "%[author_id]" is true if the item has an author_id.

  • "NOT %[author_id]" is true if the item doesn’t have an author_id.

  • "%[author_id] == 1234" is true if the item author_id is 1234.

  • "%[author_ids] == [1,2,3,4]" is true if item author_ids are 1, 2, 3, and 4.

  • "%[author_id] OR %[author_name]" is true if the item has an author_id or an author_name.

  • "%[author_id] AND %[author_name]" is true if the item has an author_id and an author_name.

  • "%[author_id] > 123" is true if the item author_id is greater than 123.

  • "(%[author_id] OR %[author_sys_id]) AND %[author_name]" is true if the item has an author_id or an author_name, as well as an author_name.

IsBinaryBody (Boolean)

Services > Endpoints > IsBinaryBody

Whether the subquery retrieves binary content as the body of the item. Default value is false.

This property is available within SubQueries objects only. It allows you to download a file (for example, XLS or PDF document) to use as the item body in a search result Quick view. See Supported file formats for an exhaustive list of item types that Coveo can index.

Example:

"SubQueries": [
{
  "Headers": {
    "x-api-key": "@ApiKey"
  },
  "Path":"%[coveo_parent.url]",
  "Method":"GET",
  "IsBinaryBody":true,
  "Body": "%[data.body]"
}
]
Notes

  • IsBinaryBody and IsThumbnail are mutually exclusive. In other words, if you set IsBinaryBody to true, you must set IsThumbnail to false, and vice versa.

  • If Coveo fails to index the item’s body, ensure that your API key is valid for subqueries, and try adding a Headers object to your subquery.

ItemPath (string, required)

Services > Endpoints > ItemPath

To index items from your application’s JSON response, Coveo needs to know where to find the items in the response.

Its value should be d.results (see response structure).

Metadata (object)

Services > Endpoints > Metadata

A key-value list of the metadata to parse.

This object maps attributes from the integration object to the Coveo fields. Some ecommerce fields are created and mapped by default. You can add customs fields via the Fields (platform-ca | platform-eu | platform-au) page.

Each key represents the metadata name of the item, while its value is the value path (simple path or JSONPath) in the JSON response.

Typically, the value path consists of one or more dynamic values, since a static, hardcoded value would result in an identical piece of metadata for all items. However, you could choose to use a hardcoded value so that the corresponding Coveo field is filled even if the API doesn’t provide this information.

Example:

"metadata": {
  "objecttype": "Product",
  "ec_brand": "%[brand]",
  "ec_name": "%[name]",
  "name": "%[name]",
  "ec_item_group_id": "%[code]",
  "ec_shortdesc": "%[summary]",
  "ec_description": "%[description]",
  "ec_price": "%[europe1Prices.results[0].price]",
  "ec_thumbnails": "https://accstorefront.commerce.ondemand.com%[thumbnail.URL]",
  "ec_in_stock": "%[stock]",
  "ec_images": "https://accstorefront.commerce.ondemand.com%[picture.URL]",
  "ec_category": "%[supercategories.results[:].name]",
  "ec_pricerange": "%[priceRange]",
  "ec_configurable": "%[configurable]",
  "ec_code": "%[code]",
  "ec_url": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
}

Method (string enum)

Services > Endpoints > Method

The HTTP method to use to fetch the resource.

Possible values are GET and POST.

This property is not inheritable.

See the API’s documentation to determine which one you should use.

Example: "Method": "POST"

ModifiedDate (string)

Services > Endpoints > ModifiedDate

The date on which the item was last modified. Although this property isn’t required in the JSON configuration, we recommend that you include it, as the Coveo-powered search pages use it for their default Sort by date option.

Typically, ModifiedDate has a dynamic value, since a static, hardcoded value would make dates identical for all items.

Example: "ModifiedDate": "%[updated]"

PayloadJsonContent (string)

Services > Endpoints > PayloadJsonContent

The JSON content to send as a POST request body. The Content-Type header is automatically set to application/json.

Dynamic values are supported when specified within the SubItems and SubQueries objects only.

In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.

Notes

  • Since the JSON content is provided as a string value in a JSON configuration, it must be escaped accordingly.

  • PayloadJsonContent and PayloadParameters are mutually exclusive. In other words, if you provide a PayloadJsonContent string, you cannot provide payload parameters, and vice versa.
Example

You want to provide the following JSON content with your request:

{
  "key1": "value1",
  "key2": "@field=(\"value2\")",
  "key3":
    {
      "subkey1": "subvalue1"
    }
}

Your source JSON configuration therefore contains the following:

"Endpoints": [
  {
    "Method": "POST",
    "Path": "/rest/myendpoint/",
    "PayloadJsonContent": "{\"key1\": \"value1\", \"key2\": \"@field=(\\\"value2\\\")\", \"key3\": {\"subkey1\": \"subvalue1\"}}"
  }
]

PayloadParameters (object)

Endpoints > PayloadParameters

A key-value list of HTTP parameters to add to the payload.

Each value can be either a number, string, or Boolean.

This property can only be used in a POST request. The parameters are sent as URL-encoded data in the request body. The Content-Type header is automatically set to application/x-www-form-urlencoded.

Dynamic values are supported when specified within the SubItems and SubQueries objects only.

Dynamic time expressions are also supported.

Notes

  • PayloadParameters and PayloadJsonContent are mutually exclusive. In other words, if you provide payload parameters, you cannot provide a JSON object, and vice versa.

  • In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.

Examples:

Basic example 
"PayloadParameters": {
  "type": "post",
  "expand": "true",
  "id": 120
}

PermanentId (string)

Services > Endpoints > PermanentId

The PermanentId is an alternative, permanent unique item identifier.

When you expect the item Uri to change, this ensures that Coveo Machine Learning (Coveo ML) recognizes the items despite their new URI and associates them to the old data. So, in the future, if you modify or delete this source and reindex the same items, specifying the same item PermanentId format in your next configuration will ensure that ML doesn’t consider your items as new and rather picks up where it left off.

Dynamic values are supported.

Example:

{
  "Uri": "%[coveo_url]/items/%[sys_id]",
  "PermanentId": "%[sys_id]"
}

ProcessingAction (object)

Services > Endpoints > ProcessingAction

This object works similarly to the IndexingAction object.

When the Condition resolves to true, the Coveo crawler applies the specified action on the corresponding child items of an item. Possible actions are Process and Ignore, that is, the crawler can either index the subitems or ignore them.

A child item ignored with ProcessingAction isn’t indexed and therefore not visible in the Content Browser or a search interface.

If the condition resolves to false, it fallbacks to the default action, that is, Process.

Example:

"ProcessingAction": {
    "ActionOnItem": "Ignore",
    "Condition": "%[id]==1"
}

Like the IndexingAction object, the ProcessingAction object supports the following properties:

QueryParameters (object)

Services > Endpoints > QueryParameters

A key-value list of HTTP parameters to add to the query.

Each value can be either a number, string, or Boolean.

Dynamic values are supported when specified within the SubItems, SubQueries, and PermissionSubQueries arrays only. They aren’t supported in QueryParameters objects of a refresh endpoint.

Note

In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.

Dynamic time expressions are also supported.

Examples:

Basic example 
"QueryParameters": {
  "type": "post",
  "expand": "true",
  "id": 120
}

Among other query parameters, the SAP source supports the following:

Filter (string)

Services > Endpoints > QueryParameters > Filter

An expression specifying the products to be indexed. It must be URL-encoded.

Example: "$filter": "catalogVersion/integrationKey eq 'Online%7CpowertoolsProductCatalog' and approvalStatus/code eq 'approved'"

Expand (string)

Services > Endpoints > QueryParameters > Expand

The list of entries associated with a single integration object or a collection of integration objects.

Example: "$expand": "supercategories,thumbnails,picture,thumbnail,unit,catalogVersion/catalog,europe1Prices,europe1Prices/currency,bundleTemplates,localizedAttributes,supercategories/localizedAttributes,approvalStatus"

RefreshEndpoints (array)

Services > Endpoints > RefreshEndpoints

The RefreshEndpoints array is required for your source to perform refresh operations, which index only the items that have been added, updated, or deleted since the last update operation. Refreshes are typically faster than rescan and rebuild operations, as they only process the items that have changed. For more information on source update operations, see Refresh, rescan, and rebuild.

Coveo recommends you configure a RefreshEndpoints array for each endpoint in your source configuration.

Each object in the RefreshEndpoints array represents a query that Coveo will make to your API when conducting a source refresh operation.

By default, a refresh object inherits the properties defined in its parent endpoint object. So, in a refresh query, you only need to specify what should be different from the original indexing query.

Then, when you refresh your source, the properties defined in an object of the RefreshEndpoints array override the properties defined under Endpoints. When you rescan or rebuild the source, however, the Endpoints properties apply.

Example

When refreshing your source, Coveo indexes only the published items that have been updated since the last refresh operation, as instructed by the following RefreshEndpoints array. Since no other property is specified, Coveo will index these items exactly like during the initial indexing operation.

"Endpoints": [
  {
    "Path": "/api/now/table/kb_knowledge",
    "Method": "GET",
    "ItemPath": "result",
    "ItemType": "kbknowledge",
    "Uri": "%[coveo_url]/kb_knowledge/%[sys_id]",
    "PermanentId": "%[sys_id]",
    "ModifiedDate": "%[sys_updated_on]",
    "ClickableUri": "%[coveo_url]/nav_to.do?uri=kb_knowledge.do?sys_id=%[sys_id]%26sysparam_view-ess",
    "Title": "%[short_description]",
    "Body": "%[text]",
    "QueryParameters": {
      "sysparm_query": "workflow_state=published"
    },
    "Metadata": {
      "short_description": "%[short_description]",
      "number": "%[number]",
      "workflow_state": "%[workflow_state]",
    },
    "RefreshEndpoints": [
      {
        "DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'", 1
        "QueryParameters": {
          "sysparm_query": "workflow_state=published^sys_updated_on>javascript:gs.dateGenerate(@RefreshDate)"
        }
      }
    ]
  }
]
1 The DateFormat property specifies the format of the date value in the query parameters.
Note

A source refresh operation will index a change in a sub-item only if its parent item has also changed.

Objects of the RefreshEndpoints array support the properties of Endpoints objects, plus the following:

Example:

"RefreshEndpoints": [
  {
    "DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
    "QueryParameters": {
      "workflow_state": "published",
      "since": "@RefreshDate"
    }
  },
  {
    "IsDeletionQuery": true,
    "DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
    "QueryParameters": {
      "lastModifiedDate": "@Now-7d",
      "status": "Deleted"
    }
  },
  {
    "IsDeletionQuery":true,
    "DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
    "Path": "/api/table/audit_delete",
    "QueryParameters":{
      "tablename": "kb_knowledge",
      "since": "@RefreshDate"
    }
  },
  {
    "IsDeletedItem": "%[is_deleted]",
    "DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
    "QueryParameters":{
      "since": "@RefreshDate"
    }
  }
]
IsDeletionQuery (Boolean)

Services > Endpoints > RefreshEndpoints > IsDeletionQuery

Whether the defined query returns items to delete from the index. Default value is false, which means that all retrieved items are added to the index or updated. If the query returns items to add, update, and delete all at once, use the IsDeletedItem property instead.

IsDeletedItem (Boolean)

Services > Endpoints > RefreshEndpoints > IsDeletedItem

When a refresh query returns items to add, update, and delete all at once, this property allows you to specify which of these items should be deleted. Default value is false, but it’s considered to be true if IsDeletionQuery is set to true.

You can provide a dynamic value (for example, %[has_been_deleted_flag]) to be evaluated for each item or a static value.

Example:

"%[is_deleted]" deletes all items that have a "is_deleted": true JSON property flagging them as deleted.

{
  "IsDeletedItem": "%[is_deleted]",
  "DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
  "QueryParameters":{
    "since": "@RefreshDate"
  }
}
DeleteChildren (Boolean)

Services > Endpoints > RefreshEndpoints > DeleteChildren

Whether to delete children when the retrieved parent item is deleted. Default value is false. If the value is set to true, child items are deleted based on their Uri.

SubItems (array)

Services > Endpoints > SubItems

A list of sub-items to retrieve. Each sub-item is represented by a configuration object under Endpoints.

An item can have many different sub-items, for example, a blog post containing comments and attachments as child items. The SubItems object establishes a parent-child relationship between items. There’s no restriction on the number of sub-items an item can have.

In addition, sub-items can also have sub-items, for example if blog post comments also have attachments. In the application JSON response, these sub-items are nested within each other, and so should be your SubItems arrays in your source configuration.

To refer to the metadata of a parent item, prefix the metadata name with coveo_parent. You may also need to use raw.

Examples:

When the sub-items to index are located under a different endpoint, you provide the Path to their location.

"SubItems": [
  {
    "Path": "/posts/%[coveo_parent.id]/comments",
    "Method": "GET",
    "ItemType": "Comment",
    "Uri": "%[coveo_url]/repository/posts/%[coveo_parent.id]/comments/%[id]",
    "ClickableUri": "%[coveo_url]/posts/%[coveo_parent.id]/comments/%[id]",
    "Title": "%[title]",
    "ModifiedDate": "%[updated]",
    "Body": "%[content]",
    "Metadata": {
      "author": "%[author]",
      "createdby" :"%[coveo_parent.author]",
      "id": "%[id]"
    }
  }
]

Alternatively, when the desired sub-items are located under the same endpoint as their parent items (for example, the child item data is nested within the parent item data), you omit the Path property and provide the sub-items' location using the ItemPath property.

"SubItems": [
  {
    "ItemPath": "/comments",
    "ItemType": "Comment",
    "Uri": "%[coveo_url]/repository/posts/%[coveo_parent.id]/comments/%[id]",
    "ClickableUri": "%[coveo_url]/posts/%[coveo_parent.id]/comments/%[id]",
    "Title": "%[title]",
    "ModifiedDate": "%[updated]",
    "Body": "%[content]",
    "Metadata": {
      "author": "%[author]",
      "createdby" :"%[coveo_parent.author]",
      "id": "%[id]"
    }
  }
]

SubQueries (array)

Services > Endpoints > SubQueries

Each object in the SubQueries array represents a subquery to execute on every item in order to fetch additional metadata and/or the item body. Subqueries are especially useful when this information is located under a different path than that of the item.

To refer to the metadata of a parent item, prefix the metadata name with coveo_parent. You may also need to use raw.

Example:

"SubQueries": [
  {
    "Path": "/posts/%[coveo_parent.id]/additionalinfo",
    "Method": "GET",
    "Body": "%[data.body]",
    "Metadata": {
      "created": "%[created]"
    }
  }
]

The subquery configuration supports the following properties from the Endpoints object configuration:

In addition, it supports the following property from the Services object configuration:

It also contains the following property:

IsThumbnail (Boolean)

Services > Endpoints > SubQueries > IsThumbnail

If the SubQueries object path leads to an image, set IsThumbnail to true to show this image as the thumbnail of an item.

The default value is false.

Supported thumbnail file formats are the following: .bmp, .emf, .exif, .gif, .icon, .jpeg, .png, .tiff, .wmf .

Notes

  • An Endpoint object can contain at most one subquery where IsThumbnail is set to true.

  • IsThumbnail and IsBinaryBody are mutually exclusive. In other words, if you set IsThumbnail to true, you must set IsBinaryBody to false, and vice versa.

Title (string)

Services > Endpoints > Title

The title of the item. Although this property isn’t required in the JSON configuration, we recommend that you include it, as the Content Browser displays its value.

Typically, Title contains a dynamic value, since a static, hardcoded value would make all item titles identical.

Example: "Title": "%[title]"

Url (string, required)

Services > Url

The Url value is the service URL of your SAP Commerce Cloud OData2 GET API.

Example: "Url": "https://api.acmestore.com/odata2webservices"

Authentication (object)

Services > Authentication

If your API service requires Coveo to authenticate, you must enter the necessary credentials in the Authentication section of the Add an SAP source panel.

Then, in your source JSON configuration, you must specify how Coveo should use these credentials to authenticate with the API. If your API uses password-based or OAuth 2.0 authentication, you must use the Authentication object, which contains the information Coveo will use to authenticate to the service.

What if my API uses API keys?

If your API uses API keys for authentication, Coveo will typically need to provide such a key in an HTTP header, as a query parameter, or as a payload parameter. You therefore don’t need an Authentication object in your source JSON configuration.

In the Authentication object, you’ll use a placeholder starting with @ to refer to the credentials you entered in the panel. This ensures that your credentials aren’t stored in clear text in the source JSON configuration, where other Administration Console users could see them. Then, when Coveo processes the source configuration, it replaces the placeholders with the actual credentials you entered in the panel.

The following table shows the objects and placeholders you must include in your source configuration, depending on the type of authentication your API uses:

Authentication type Use placeholders in Placeholders to use

Basic, HTTP, Kerberos, or NTLM (password-based)

Authentication object

@Username and @Password

OAuth 2.0

Authentication object

@ClientId, @ClientSecret, and @RefreshToken

API key

Headers, QueryParameters, or PayloadParameters object

@ApiKey

As indicated in the table above, API key placeholders don’t appear in the Authentication object. If your API authenticates with API keys, see the sections listed in the table for examples with API key placeholders.

Authentication properties specified at the service level apply to the child endpoints, and sub-queries underneath them. Typically, you don’t need to override the configuration you entered at the service level further down in the JSON configuration, as the same authentication method applies to the entire application content.

Examples:

Basic, HTTP, Kerberos, or NTLM

If your API uses password-based authentication, you must enter the username and password in the Authentication section of the Add an SAP source panel.

Username and password in the source panel

Then, in your source JSON configuration, you must use the @Username and @Password placeholders to refer to these credentials. Your Authentication object therefore looks like this:

"Authentication": {
  "Username": "@Username",
  "Password": "@Password",
  "ForceBasicAuthentication": "true"
}
OAuth 2.0

If your API uses OAuth 2.0 authentication, you must enter the client ID, client secret, and refresh token in the Authentication section of the Add an SAP source panel.

OAuth information in the source panel

Then, in your source JSON configuration, you must use the @ClientId, @ClientSecret, and @RefreshToken placeholders to refer to this information. Your Authentication object could therefore look like this:

"Authentication": {
 "OAuth": {
   "Query": {
     "RefreshUrl": "http://example.com/token",
     "Method": "POST",
     "Parameters": {
       "grant_type": {
         "Type": "Payload",
         "Value": "refresh_token"
       },
       "client_id": {
         "Type": "Payload",
         "Value": "@ClientId"
       },
       "client_secret": {
         "Type": "Payload",
         "Value": "@ClientSecret"
       },
       "refresh_token": {
         "Type": "Payload",
         "Value": "@RefreshToken",
         "IsRefreshToken": true
       }
     }
  }
 }
}

The Authentication object supports the following properties:

Username (string)

Services > Authentication > Username

Username used to log in to the service. You can use @Username to retrieve the value specified in the Add an SAP Source panel.

Password (string)

Services > Authentication > Password

Password used to log in to the service. This property can be empty even though a Username is specified. You can use @Password to retrieve the value specified in the Add an SAP Source panel.

Domain (string)

Services > Authentication > Domain

Domain name to use when authenticating to a NTLM or Kerberos protected server.

ForceBasicAuthentication (Boolean)

Services > Authentication > ForceBasicAuthentication

Whether to force a basic HTTP header in the request. The default value is false.

OAuth (object)

Services > Authentication > OAuth

If your application requires OAuth 2.0 authentication, add an OAuth object to your source configuration.

Your OAuth object must contain a Query object. Coveo will use the information in the Query object to make an authentication query to your application.

Optionally, your OAuth object may also contain a Response object. In this object, you may include properties such as AccessToken, TokenType, and ExpiresIn, whose values indicate where Coveo will find the corresponding information when parsing the API’s response. The Response object may also include the AuthorizationHeader property, whose value is the name of the authorization header that Coveo should use when querying your API for content.

Tip

Enter your client ID, client secret, and refresh token in the Add an SAP source panel and use the @ClientId, @ClientSecret, and @RefreshToken placeholders in your JSON configuration.

This allows you to keep your confidential information obfuscated, thus preventing other Administration Console users to access it.

Refresh token placeholder in the source JSON configuration and in the source panel

Example:

"OAuth": {
  "Query": {
    "RefreshUrl": "http://example.com/token",
    "Method": "POST",
    "Parameters": {
      "grant_type": {
        "Type": "Payload",
        "Value": "refresh_token"
      },
      "refresh_token": {
        "Type": "Payload",
        "Value": "@RefreshToken",
        "IsRefreshToken": true
      },
      "client_id": {
        "Type": "Payload",
        "Value": "@ClientId"
      },
      "client_secret": {
        "Type": "Payload",
        "Value": "@ClientSecret"
      }
    }
  },
  "Response": {
    "AccessToken": "access_token",
    "SupportsRefreshToken" : "true",
    "RefreshToken": "refresh_token",
    "ExpiresIn": "expires_in",
    "TokenType": "token_type"
  }
}

The OAuth object supports the following properties:

Query (object, required)

Services > Authentication > OAuth > Query

If your application requires OAuth 2.0 authentication, you must add an OAuth object to your source configuration.

Your OAuth object must contain a Query object. Coveo will use the information in the Query object to make an authentication query to your application.

Tip

Enter your client ID, client secret, and refresh token in the Add an SAP source panel and use the @ClientId, @ClientSecret, and @RefreshToken placeholders in your JSON configuration.

This allows you to keep your confidential information obfuscated, thus preventing other Administration Console users to access it.

Refresh token placeholder in the source JSON configuration and in the source panel

Example:

"Query": {
  "RefreshUrl": "http://example.com/token",
  "Method": "POST",
  "Parameters": {
    "grant_type": {
      "Type": "Payload",
      "Value": "refresh_token"
    },
    "refresh_token": {
      "Type": "Payload",
      "Value": "@RefreshToken",
      "IsRefreshToken": true
    },
    "client_id": {
      "Type": "Payload",
      "Value": "@ClientId"
    },
    "client_secret": {
      "Type": "Payload",
      "Value": "@ClientSecret"
    }
  }
}

The Query object supports the following properties:

RefreshUrl (string, required)

Services > Authentication > OAuth > Query > RefreshUrl

If your application requires OAuth 2.0 authentication, your Query object must include a RefreshUrl property. The RefreshUrl value must be a URL that Coveo will use to refresh its access token.

Example 
"Query": {
  "RefreshUrl": "http://example.com/token",
  "Method": "POST",
  "Parameters": {
    "grant_type": {
      "Type": "Payload",
      "Value": "refresh_token"
    },
    "refresh_token": {
      "Type": "Payload",
      "Value": "@RefreshToken",
      "IsRefreshToken": true
    },
    "client_id": {
      "Type": "Payload",
      "Value": "@ClientId"
    },
    "client_secret": {
      "Type": "Payload",
      "Value": "@ClientSecret"
    }
  }
}
OAuth query Headers (object)

Services > Authentication > OAuth > Query > Headers

If your application requires OAuth 2.0 authentication, your Query object can optionally include a Headers object.

When making an authentication query to your application, Coveo will add these HTTP headers to its request.

Note

Coveo automatically sets the Content-Type header. Therefore, you don’t need to add it to the Headers object.

Example:

"Query": {
  "RefreshUrl": "http://example.com/token",
  "Headers": {
    "X-Auth": "apikey1234567890abcdef"
  }
}
OAuth query Method (string enum)

Services > Authentication > OAuth > Query > Method

If your application requires OAuth 2.0 authentication, your Query object can optionally include a Method property.

The Method property specifies the HTTP method that Coveo should use when refreshing its token. Default is POST.

Example 
"Query": {
  "RefreshUrl": "http://example.com/token",
  "Method": "POST",
  "Parameters": {
    "grant_type": {
      "Type": "Payload",
      "Value": "refresh_token"
    },
    "refresh_token": {
      "Type": "Payload",
      "Value": "@RefreshToken",
      "IsRefreshToken": true
    },
    "client_id": {
      "Type": "Payload",
      "Value": "@ClientId"
    },
    "client_secret": {
      "Type": "Payload",
      "Value": "@ClientSecret"
    }
  }
}
OAuth query Parameters (object)

Services > Authentication > OAuth > Query > Parameters

If your application requires OAuth 2.0 authentication, your Query object can optionally include query parameters for Coveo to use.

Each object in the Parameters object represents a parameter to include in Coveo’s authentication request to your application. Each object should contain a "Type" and a "Value" property.

The Type property indicates how to include the parameter in the request. Possible values for this property are query or payload.

If you include a refresh_token object, you must add a IsRefreshToken property with a value of true to this object. Moreover, if this refresh token is single-use, make sure to specify it by adding "UsingSingleUseRefreshToken": true to your OAuth object.

Tip

Enter your client ID, client secret, and refresh token in the Add an SAP source panel and use the @ClientId, @ClientSecret, and @RefreshToken placeholders in your JSON configuration.

This allows you to keep your confidential information obfuscated, thus preventing other Administration Console users to access it.

Refresh token placeholder in the source JSON configuration and in the source panel

Example:

"Authentication": {
  "OAuth": {
    "UsingSingleUseRefreshToken": true,
    "Query": {
      "RefreshUrl": "https://auth.example.com/oauth/token",
      "Method": "POST",
      "Parameters": {
        "grant_type": {
          "Type": "Payload",
          "Value": "refresh_token"
        },
        "refresh_token": {
          "Type": "Payload",
          "Value": "@RefreshToken",
          "IsRefreshToken": true
        },
        "client_id": {
          "Type": "Payload",
          "Value": "@ClientId"
        },
        "client_secret": {
          "Type": "Payload",
          "Value": "@ClientSecret"
        }
      }
    }
  }
}
Response (object)

Services > Authentication > OAuth > Response

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.

The Response object contains, among other things, the properties that Coveo must extract from the authentication server’s response. The value of each property represents the path to the corresponding value in the response.

In the Response object, you may also enter other authentication parameters, such as the AuthorizationHeader and SupportsRefreshToken properties.

Example:

If the authentication server’s response looks like this:

HTTP/1.1 200 OK
Content-Type: application/json
{
  "access_token": "jOs0PoLKm61sd6h49",
  "refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
  "token_type": "bearer",
  "expires_in": 86400
}

Then your Response object should look like this:

"Response": {
  "AccessToken": "access_token",
  "SupportsRefreshToken" : "true",
  "RefreshToken": "refresh_token",
  "ExpiresIn": "expires_in",
  "TokenType": "token_type"
}

The Response object supports the following properties:

AccessToken (string)

Services > Authentication > OAuth > Response > AccessToken

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object. Coveo will then extract the information from the API’s response using the values specified in the Response object.

In the Response object, you can add the AccessToken property to specify the path to the access token in the API’s response. Default is access_token.

Example

If the authentication server’s response looks like this:

HTTP/1.1 200 OK
Content-Type: application/json
{
  "access_token": "jOs0PoLKm61sd6h49",
  "refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
  "token_type": "bearer",
  "expires_in": 86400
}

Then your Response object should look like this:

"Response": {
  "AccessToken": "access_token",
  "SupportsRefreshToken" : "true",
  "RefreshToken": "refresh_token",
  "ExpiresIn": "expires_in",
  "TokenType": "token_type"
}
SupportsRefreshToken (Boolean)

Services > Authentication > OAuth > Response > SupportsRefreshToken

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.

In the Response object, you can add the SupportsRefreshToken property to indicate whether your application’s OAuth 2.0 flow supports refresh tokens. Default is true.

Example

If the authentication server’s response looks like this:

HTTP/1.1 200 OK
Content-Type: application/json
{
  "access_token": "jOs0PoLKm61sd6h49",
  "refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
  "token_type": "bearer",
  "expires_in": 86400
}

Then your Response object should look like this:

"Response": {
  "AccessToken": "access_token",
  "SupportsRefreshToken" : "true",
  "RefreshToken": "refresh_token",
  "ExpiresIn": "expires_in",
  "TokenType": "token_type"
}
RefreshToken (string)

Services > Authentication > OAuth > Response > RefreshToken

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object. Coveo will then extract the information from the API’s response using the values specified in the Response object.

In the Response object, you can add the AccessToken property to specify the path to the refresh token in the API’s response. Default is refresh_token.

Example

If the authentication server’s response looks like this:

HTTP/1.1 200 OK
Content-Type: application/json
{
  "access_token": "jOs0PoLKm61sd6h49",
  "refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
  "token_type": "bearer",
  "expires_in": 86400
}

Then your Response object should look like this:

"Response": {
  "AccessToken": "access_token",
  "SupportsRefreshToken" : "true",
  "RefreshToken": "refresh_token",
  "ExpiresIn": "expires_in",
  "TokenType": "token_type"
}
ExpiresIn (string)

Services > Authentication > OAuth > Response > ExpiresIn

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object. Coveo will then extract the information from the API’s response using the values specified in the Response object.

In the Response object, you can add the ExpiresIn property to specify the path to the refresh token expiration delay in the API’s response. Default is expires_in.

If the API’s response doesn’t contain an expiration delay, use the ExpiresInDefaultValue property instead.

Example

If the authentication server’s response looks like this:

HTTP/1.1 200 OK
Content-Type: application/json
{
  "access_token": "jOs0PoLKm61sd6h49",
  "refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
  "token_type": "bearer",
  "expires_in": 86400
}

Then your Response object should look like this:

"Response": {
  "AccessToken": "access_token",
  "SupportsRefreshToken" : "true",
  "RefreshToken": "refresh_token",
  "ExpiresIn": "expires_in",
  "TokenType": "token_type"
}
ExpiresInDefaultValue (number)

Services > Authentication > OAuth > Response > ExpiresInDefaultValue

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object. Coveo will then extract the information from the API’s response using the values specified in the Response object.

Typically, the authentication server will return an expiration delay for its refresh token, which you can extract with the ExpiresIn property. However, if the API’s response doesn’t include an expiration delay, you can add the ExpiresInDefaultValue property to your Response object.

The ExpiresInDefaultValue value specifies the number of seconds after which Coveo should consider the refresh token expired. Default is 3600.

Example:

"Response": {
  "AccessToken": "access_token",
  "SupportsRefreshToken" : "true",
  "RefreshToken": "refresh_token",
  "ExpiresInDefaultValue": 1800,
  "TokenType": "token_type"
}
TokenType (string)

Services > Authentication > OAuth > Response > TokenType

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object. Coveo will then extract the information from the API’s response using the values specified in the Response object.

In the Response object, you can add the ExpiresIn property to specify the path to the token type in the API’s response. Default is token_type.

If the response doesn’t specify a token type or if it’s invalid or inappropriate, Coveo will use Bearer by default. If you want a different default value, use the TokenTypeDefaultValue property instead.

Example

If the authentication server’s response looks like this:

HTTP/1.1 200 OK
Content-Type: application/json
{
  "access_token": "jOs0PoLKm61sd6h49",
  "refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
  "token_type": "bearer",
  "expires_in": 86400
}

Then your Response object should look like this:

"Response": {
  "AccessToken": "access_token",
  "SupportsRefreshToken" : "true",
  "RefreshToken": "refresh_token",
  "ExpiresIn": "expires_in",
  "TokenType": "token_type"
}
TokenTypeDefaultValue (string)

Services > Authentication > OAuth > Response > TokenTypeDefaultValue

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object. Coveo will then extract the information from the API’s response using the values specified in the Response object.

Typically, the authentication server will return a token type, which you can extract with the TokenType property. If the API’s response doesn’t specify a token type, or if the token type is invalid or inappropriate, Coveo will use Bearer by default.

However, if you want Coveo to use a different default token type, you can specify it by the TokenTypeDefaultValue property to your Response object.

AuthorizationHeader (string)

Services > Authentication > OAuth > Response > AuthorizationHeader

If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.

In the Response object, you can add the AuthorizationHeader property to indicate the name of the authorization header that Coveo should use to provide its access token when querying your API for content.

Refer to your API’s documentation for the header name to use. Default is Authorization.

Notes

Coveo automatically adds the specified authorization header to its content requests to your application. Do not add this header to the Headers object of your service.

Example:

If your API documentation specifies that the access token must be provided in a header named X-ACCESS-TOKEN, then your Response object should specify it like this:

"Response": {
  "AccessToken": "access_token",
  "SupportsRefreshToken" : "true",
  "RefreshToken": "refresh_token",
  "ExpiresIn": "expires_in",
  "TokenType": "token_type",
  "AuthorizationHeader": "X-ACCESS-TOKEN"
}

As a result, when making content requests to your application, Coveo will automatically include an X-ACCESS-TOKEN header with the access token value.

{
  "headers": {
    "Content-Type": "application/json",
    "X-ACCESS-TOKEN": "123456789abcdef"
  }
}
UsingSingleUseRefreshToken (Boolean)

Set UsingSingleUseRefreshToken to true to indicate that the refresh token used by the source to authenticate to your application is single-use. Default is false. A single-use refresh token expires after it’s been used.

When setting UsingSingleUseRefreshToken to true, make sure to include a refresh_token query parameter in your OAuth query.

Example 
"Authentication": {
  "OAuth": {
    "UsingSingleUseRefreshToken": true,
    "Query": {
      "RefreshUrl": "https://auth.example.com/oauth/token",
      "Method": "POST",
      "Parameters": {
        "grant_type": {
          "Type": "Payload",
          "Value": "refresh_token"
        },
        "client_id": {
          "Type": "Payload",
          "Value": "@ClientId"
        },
        "client_secret": {
          "Type": "Payload",
          "Value": "@ClientSecret"
        },
        "refresh_token": {
          "Type": "Payload",
          "Value": "@RefreshToken",
          "IsRefreshToken": true
        }
      }
    }
  }
}

Headers (object)

Services > Headers

A key-value list of HTTP headers to add to the query. Each value can be either a number, string, or Boolean.

Your Headers object must include a unique Application Interface Key (Application Interface Register, AIR). Refer to the SAP documentation for details.

The Headers object is inheritable.

Examples:

Basic example 
"Headers": {
  "Custom-Header-Name": "myValue",
  "Application-Interface-Key": "xxxxxxxx"
}

Paging (object)

Services > Paging

In the Paging object, specify how you want the content to be paged.

This property is inheritable, but you can disable inheritance using the DoNotInherit property. This may help speed up the crawling process.

When you set it at the service level, it will apply to all endpoints in this service by default. You can override the service-level configuration by setting the Paging object again at the endpoint level. Sub-items inherit this property. Sub-queries don’t inherit this property.

Example:

"Paging": {
  "PageSize": 20,
  "OffsetStart": 0,
  "OffsetType": "page",
  "Parameters": {
    "Limit": "limit",
    "Offset": "page"
  },
  "DoNotInherit": true
}

The Paging object supports the following properties:

OffsetType (string enum, required)

Services > Paging > OffsetType

When configuring your source to request paginated content, you must specify how your API paginates its content. Coveo needs this information to build its page request URLs in the appropriate format. You can typically find it in your API’s documentation.

The OffsetType property is a string enum that specifies the type of pagination your API uses. The value to specify depends on your API’s pagination technique, just like the properties you’ll need to add to your Paging object.

Pagination technique OffsetType value Properties to add to the Paging object

Page-based

page

OffsetStart, PageSize, and paging Parameters

Offset-based

item

OffsetStart, PageSize, and paging Parameters

URL-based

url

PageSize, NextPageKey, and paging Parameters

Cursor-based

cursor

PageSize, NextPageKey, and paging Parameters

Examples:

Page-based pagination

Your API expects request URLs in a format such as http://example.com/api/item?limit=100&pageNumber=0 for the first page, and http://example.com/api/item?limit=100&pageNumber=1 for the second page. With these URLs, Coveo would request the first page (page 0), and then increment the page number by 1 to request the second page, and so on.

In this case, your Paging object should look like this:

{
  "Paging": {
    "PageSize": 100,
    "OffsetStart": 0,
    "OffsetType": "page",
    "Parameters": {
        "Limit": "limit",
        "Offset": "pageNumber"
    }
  }
}
Offset-based pagination

Your API expects request URLs in a format such as http://example.com/api/item?limit=100&start=0 for the first page, and http://example.com/api/item?limit=100&start=100 for the second page. With these URLs, Coveo would request the 100 first items starting from item 0, and then the 100 next items, starting from item 100.

In this case, your Paging object should look like this:

{
  "Paging": {
    "PageSize": 100,
    "OffsetStart": 0,
    "OffsetType": "item",
    "Parameters": {
        "Limit": "limit",
        "Offset": "start"
    }
  }
}
URL-based pagination

Your API expects request URLs in a format such as http://example.com/api/item?limit=8 for the first page, and http://example.com/api/item?page=3d170d80d8n3n2342c328s for the second page. In this context, Coveo would request the first page, and then the next page using the URL provided in the nextPage value in the JSON response.

In this case, if your API returns the following first page:

{
  "nextPage": "http://example.com/api/item?page=3d170d80d8n3n2342c328s",
  "items": [
    { "id": 1, "name": "Caroline" },
    { "id": 2, "name": "Marcella" },
    { "id": 3, "name": "Susie" },
    { "id": 4, "name": "Rhonda" },
    { "id": 5, "name": "Wendy" },
    { "id": 6, "name": "Barbara" },
    { "id": 7, "name": "Deirdre" },
    { "id": 8, "name": "Lynda" }
  ]
}

Then your Paging object should look like this:

{
  "Paging": {
    "PageSize": 8,
    "NextPageKey": "nextPage",
    "OffsetType": "url",
    "Parameters": {
        "Limit": "limit"
    }
  }
}
Cursor-based pagination

Your API expects request URLs in a format such as http://example.com/api/item?limit=8 for the first page, and http://example.com/api/item?limit=8&nextPageToken=3d170d80-7b3b-4377 for the second page. With these URLs, Coveo would request the 8 first items, and then the next page using the token provided in the NextPageId value in the JSON response.

In this case, if your API returns the following first page:

{
  "nextPageId": "3d170d80-7b3b-4377",
  "items": [
    { "id": 1, "name": "Caroline" },
    { "id": 2, "name": "Marcella" },
    { "id": 3, "name": "Susie" },
    { "id": 4, "name": "Rhonda" },
    { "id": 5, "name": "Wendy" },
    { "id": 6, "name": "Barbara" },
    { "id": 7, "name": "Deirdre" },
    { "id": 8, "name": "Lynda" }
  ]
}

Then your Paging object should look like this:

{
  "Paging": {
    "PageSize": 8,
    "NextPageKey": "NextPageId",
    "OffsetType": "cursor",
    "Parameters": {
        "Limit": "limit",
        "Offset": "nextPageToken"
    }
  }
}

PageSize (number, required)

Services > Paging > PageSize

The number of items to fetch per page.

Example: With paging URL https://example.com/api/item?position=0&quantity=50, your JSON configuration should include "PageSize": 50.

DoNotInherit (Boolean)

Services > Paging > DoNotInherit

Whether to prevent your Paging configuration from being inherited. This can be useful for preventing paging on PermissionSubqueries, which is known to slow down the crawling process.

The default value is false.

NextPageKey (string)

Services > Paging > NextPageKey

When the OffsetType value is "url" or "cursor", provide the path (simple path or JSONPath) to the key representing the value of the next page reference, that is, either the next page URL or the next page cursor.

The NextPageKey property isn’t required when the OffsetType value is page or item.

Example: "NextPageKey": "NextPageLink"

OffsetStart (number)

Services > Paging > OffsetStart

Offset of the first page to fetch. This property is required when the OffsetType value is "page" or "item".

Example: Your first paging URL is https://example.com/api/item?position=0&quantity=50. If you want to retrieve all items except items 1 to 50, your paging configuration must include "OffsetStart": 1.

Paging Parameters (object)

Services > Paging > Parameters

In the paging Parameters object, you must provide the names of the query string fields in your web application pagination URL that are required to support pagination. The properties expected in this object are Limit and Offset.

However, Offset isn’t required when the OffsetType value is "url" since the complete URL is already retrieved in the JSON response.

Example:

"Parameters": {
  "Limit": "limit",
  "Offset": "page"
}

The Parameters object supports the following properties:

Limit (string, required)

Services > Paging > Parameters > Limit

In the query string, the field whose value is retrieved with the PageSize parameter.

Example: With paging URL https://example.com/api/item?position=0&quantity=50, your JSON configuration should include "Limit": "quantity".

FetchNextPageUntilNoResult (Boolean)

Services > Paging > Parameters > FetchNextPageUntilNoResult

The FetchNextPageUntilNoResult property indicates whether Coveo will try to fetch pages until no result is returned, or until the application API returns an error. Default value is true.

The FetchNextPageUntilNoResult property is an alternative to the TotalCountHeaderKey and TotalCountKey properties.

Tip

Unless you set this property to false, we recommend also using the SkippableErrorCodes property at the service and/or endpoint level. This ensures that the crawling process continues when the API returns an error.

Example:

"Paging": {
  "OffsetType": "page",
  "PageSize": 25,
  "Parameters": {
    "FetchNextPageUntilNoResult": true
  }
}
Offset (string)

Services > Paging > Parameters > Offset

In the query string, the field whose value is retrieved with the OffsetStart property.

Example: Your first paging URL is https://example.com/api/item?position=0&quantity=50. If you want to ignore items 1 to 50 and retrieve items 51 and over, your paging configuration must include "Offset": "position" and "OffsetStart": 1.

TotalCountHeaderKey (string)

Services > Paging > TotalCountHeaderKey

The name of an HTTP header whose value represents the total number of items retrieved through the API call. This property is useful when Coveo makes an API call to a page that doesn’t exist, causing an error to be returned.

Example: "TotalCountHeaderKey": "x-total-count"

TotalCountKey (string)

Services > Paging > TotalCountKey

The path (simple path or JSONPath) to a response body property representing the total number of items retrieved through the API call. This property is useful when Coveo makes an API call to a page that doesn’t exist, causing an error to be returned.

Example: "TotalCountKey": "retrievedContent.totalCount"

Permissions (array)

Services > Permissions

Each object in the Permissions array represents a permission level. In turn, permission levels contain one or more PermissionsSets arrays.

The Permissions array is required when you select the Same users and groups as in your content system source content security option. It contains the permission levels and permission sets to apply to the items indexed through this service. The members listed in these levels and sets are either allowed or denied access to the indexed items.

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

This property is inheritable.

Example:

"Permissions": [
  {
    "Name": "Permission Level 1",
    "PermissionsSets": [
      {
        "Name": "Permission Set 1",
        "AllowedMembers": [
          {
            "Name": "jsmith@example.com",
            "Type": "User",
            "AdditionalInfo": {
              "title": "%[customMetadata]"
            }
          }
        ],
        "DeniedMembers": null,
        "IsAnonymousAllowed": false
      }
    ]
  }
]

The Permissions array supports the following properties:

Permission level Name (string)

Services > Permissions > Name

The name of the permission level.

PermissionsSets (array)

Services > Permissions > PermissionsSets

The PermissionsSets array specifies who can or can’t access the indexed items. Each object in the array describes a single permission set, which can contain lists of allowed and denied members.

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

Objects in the PermissionsSets array support the following properties:

Permission set Name (string)

Services > Permissions > PermissionsSets > Name

The Name property indicates the name of the permission set.

AllowedMembers and DeniedMembers arrays

Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers

Each object in the AllowedMembers and DeniedMembers arrays represents a member security identity that should be allowed or denied to access the indexed content.

Note

If a user is listed under both AllowedMembers and DeniedMembers for an item, the denial prevails so that security holes are avoided.

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

Example

You index text items written by different people. The metadata of each item contains the key author, and, in this system, authors are identified by their email address. These addresses are therefore expected as values of the Name field. With the following configuration, each user in your organization can access the items they have created through your Coveo-powered search interfaces.

"PermissionsSets": [
  {
    "Name": "Permission Set 1",
    "AllowedMembers": [
      {
        "Name": "%[author]",
        "Type": "User",
        "PermissionType": "config1",
        "Condition": "%[field] == value",
        "AdditionalInfo": {
          "title": "%[customMetadata]"
        }
      }
    ],
    "DeniedMembers": null,
    "IsAnonymousAllowed": false
  }
]

AllowedMembers and DeniedMembers arrays support the following properties:

Member Name (string, required)

Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > Name

The name of the member security identity.

Email addresses and Active Directory usernames are supported.

Dynamic values are supported.

Member Type (string enum, required)

Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > Type

The type of member. Allowed values are User, Group, and VirtualGroup.

For more information on these types of security identities, see Coveo management of security identities and item permissions.

Permission Condition (string)

Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > Condition

The Condition value is a condition that must resolve to true for the security identity specified in Name to be resolved, that is, extracted by the security identity provider. As a result, the permission applies.

When the condition resolves to false, the security identity is not resolved and the corresponding permission doesn’t apply.

Adding a condition to your AllowedMembers or DeniedMembers object is optional.

To write your condition, use the same syntax as for indexing pipeline extension conditions. Conditions can be assembled using the following operators: AND, OR, Exists, NOT, >, and <. Parentheses are also supported to specify operation order.

The metadata fields in your condition must either be defined in the Metadata object of the source configuration or referenced with raw. You can specify an array if the metadata refers to a multi-value field in your repository. Dynamic values are also supported.

Example

You index a list of available and discontinued products, and your public search interface is used by both your employees and your customers. On this search interface, your employees can log in to see additional content, while customers can’t log in.

You want to hide your discontinued products from your customers only, so you use the Condition property to apply a permission to these items. As a result, all products with Discontinued: yes in their metadata are visible to members of the *@mycompany.com group only, that is, your employees, while non-discontinued products are available to all search interface users.

"PermissionsSets": [
  {
    "Name": "Permission Set 1",
    "AllowedMembers": [
      {
        "Name": "*@mycompany.com",
        "Type": "Group",
        "PermissionType": "config1",
        "Condition": "%[discontinued] == yes"
      }
    ],
    "DeniedMembers": null,
    "IsAnonymousAllowed": false
  }
]

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

PermissionType (string)

Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > PermissionType

This property is relevant only when you provide a permission configuration.

The PermissionType is a key representing the configuration on how to extract all relationships of the security identity specified in Name. The security identity provider will use this configuration when processing the security identity.

In the Content Security tab of the Add/Edit an SAP Source panel, you’ll need to select the Same users and groups as in your content system, and then to provide a JSON permission configuration detailing how to retrieve the relationships of each security identity and how to index this data.

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

AdditionalInfo (object)

Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > AdditionalInfo

The AdditionalInfo object is relevant only when you provide a permission configuration.

It is a key-value store of additional information required by some security providers. Each key represents the metadata name of the repository item, while its value is the value path (simple path or JSONPath) in the JSON response.

When writing your permission configuration, you can use coveo_parent to refer to the information retrieved with the AdditionalInfo object of a permission subquery.

Dynamic values are supported in the values of this object.

IsAnonymousAllowed (Boolean)

Permissions > PermissionsSets > IsAnonymousAllowed

The IsAnonymousAllowed property indicates whether anonymous, that is, unauthenticated users are allowed to access the retrieved items. Default value is false.

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

PermissionsFromMetadata array

Services > Permissions > PermissionsSets > PermissionsFromMetadata

This array is relevant only when you provide a permission configuration.

The PermissionsFromMetadata array is an alternative to the PermissionSubQueries array, as it fetches permission data, typically the security identities that are allowed or denied access to the item. This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo.

Include a PermissionsFromMetadata array in your source configuration when the permission data appears in the metadata of an item. Conversely, a PermissionSubQueries array should be used when an item and its permission data must be retrieved with two different queries.

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

Objects of the PermissionsFromMetadata array support:

PermissionSubQueries array

Services > Permissions > PermissionsSets > PermissionSubQueries

This array is relevant only when you provide a permission configuration.

Each object in the PermissionSubQueries array represents a subquery to execute on every item to fetch permission data, typically the security identities that are allowed or denied access to the item. This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo. Each object in the PermissionSubQueries array inherits the paging properties specified at the parent endpoint or service level.

The PermissionSubQueries array is an alternative to the PermissionsFromMetadata array.

Include a PermissionSubQueries array in your source configuration when an item and its permission data must be retrieved with two different queries. Conversely, the PermissionsFromMetadata array should be used when the permission data appears in the metadata of an item.

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

Objects of the PermissionSubQueries array support:

Example:

"permissionSubQueries": [
  {
    "ItemPath": "permissions",
    "Path": "%[productid]/permissions",
    "Method": "GET",
    "QueryParameters": {},
    "Name": "%[name]",
    "Type": "group",
    "PermissionType": "config1",
    "Condition": "%[field] == value",
    "IsAllowedMember": true,
    "AdditionalInfo":
      {
        "PermissionID": "%[id]",
        "PermissionName": "%[name]"
      }
  }
]
IsAllowedMember (Boolean)

Services > Permissions > PermissionsSets > PermissionSubQueries > IsAllowedMember

The IsAllowedMember property indicates whether the specified member is allowed to access the item. Default value is false.

For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.

SkippableErrorCodes (string)

Services > SkippableErrorCodes

When crawling your content, Coveo may encounter an HTTP error and stop. If you expect the web application API to return an error and want it to be ignored so that the crawling process continues, specify the error code as the SkippableErrorCodes property value.

This property is inheritable. When you set it at the service level, it will apply to all endpoints in this service, and subqueries in these endpoints. You can override the service-level configuration by setting the SkippableErrorCodes property again at the endpoint or subquery level. Sub-items don’t inherit this property.

Use a semicolon (;) to separate error codes.

Example: "SkippableErrorCodes": "500;404;403"