SAP Source Reference
SAP Source Reference
- Service Object
- Endpoint Object
- Action Object
- Member Object
- Paging Object
- Parameter Object
- Permission Level Object
- Permission Set Object
- PermissionsFromMetadata Object
- Permission Subquery Object
- Refresh Endpoint Object
- Subquery Object
- Authentication Object
- OAuth Object
- OAuth Query Object
- OAuth Response Object
When creating an SAP source in the Coveo Administration Console, you must provide a JSON configuration detailing which services and endpoints to crawl to retrieve the desired content, and how to retrieve items of each type.
This configuration has a single property, Services
, whose value must be an array of Service Object.
{
"services": [
{
"url": "https://api.acmestore.com/odata2webservices",
"headers": {
"Accept": "application/json",
"Application-Interface-Key": "xxxxxxxx"
},
"paging": {
"pageSize": 10,
"offsetStart": 0,
"offsetType": "item",
"nextPageKey": "__next",
"parameters": {
"limit": "$top",
"offset": "$skip"
}
},
"endpoints": [
{
"path": "/Product/Products",
"method": "GET",
"queryParameters": {
"$filter": "catalogVersion/integrationKey eq 'Online%7CpowertoolsProductCatalog' and approvalStatus/code eq 'approved'",
"$expand": "supercategories,thumbnails,picture,thumbnail,unit,catalogVersion/catalog,europe1Prices,europe1Prices/currency,bundleTemplates,localizedAttributes,supercategories/localizedAttributes,approvalStatus"
},
"itemPath": "d.results",
"itemType": "product",
"uri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]",
"clickableUri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]",
"metadata": {
"objecttype": "Product",
"ec_brand": "%[brand]",
"ec_name": "%[name]",
"name": "%[name]",
"ec_item_group_id": "%[code]",
"ec_shortdesc": "%[summary]",
"ec_description": "%[description]",
"ec_price": "%[europe1Prices.results[0].price]",
"ec_thumbnails": "https://accstorefront.commerce.ondemand.com%[thumbnail.URL]",
"ec_in_stock": "%[stock]",
"ec_images": "https://accstorefront.commerce.ondemand.com%[picture.URL]",
"ec_category": "%[supercategories.results[:].name]",
"ec_pricerange": "%[priceRange]",
"ec_configurable": "%[configurable]",
"ec_code": "%[code]",
"ec_url": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
}
}
]
}
]
}
This reference article defines parameters to include in your JSON configuration.
When building your JSON configuration, you may also want to refer to the SAP Source Configuration Reference and the Concepts article.
Service Object
Each object in the Services
array specifies a REST API service to retrieve content through, along with the specific endpoints to call.
The properties expected in an object of the Services
array are the following.
Endpoints (Array of Endpoint Object, Required)
Each object in the Endpoints
array details how you want Coveo to crawl an endpoint of your SAP Commerce instance.
You can specify as many endpoints as you need.
See the Endpoints section for details on the Endpoint
object properties.
By default, the SAP source JSON configuration contains three endpoints, one for:
-
product (e.g.,
"path": "/Product/Products"
) -
variant (e.g.,
"path": "FootWearVariant/GenericVariantProducts"
) -
availability (e.g.,
"path": "/Availability/Warehouses"
)
You can remove or add endpoints depending on your use case.
Url (String, Required)
The Url
value is the URL of your SAP Commerce Cloud OData2 GET API.
Example: "Url": "https://api.acmestore.com/odata2webservices"
Headers (Object)
A key-value list of HTTP headers to add to the query. Each value can be either a number, string, or Boolean.
|
AIR is required
You have to specify a unique Application Interface Key (Application Interface Register, AIR). Refer to the SAP documentation for details. |
This property is inheritable.
Example:
"headers": {
"Custom-Header-Name": "myValue",
"Application-Interface-Key": "xxxxxxxx"
}
Paging (Paging Object)
In the Paging
object, you can specify how you want the Services
content to be paged.
See the Paging section for details on the properties in this object.
Permissions (Array of Permission Level Object)
The Permissions
array is required when you select the Same users and groups as in your current permission system source content security option.
It contains the permission levels and sets to apply to the items indexed through this service.
The members listed in these levels and sets are either allowed or denied access to the indexed items.
This property is inheritable.
See the Permission Level Object section for details on the properties in this object.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
SkippableErrorCodes (String)
When crawling your content, Coveo may encounter an HTTP error and stop.
If you expect the web application API to return an error and want it to be ignored so that the crawling process continues, specify the error code as the SkippableErrorCodes
property value.
This property applies at the Services
level and is inheritable.
Use a semicolon (;
) to separate error codes.
Example: "SkippableErrorCodes": "500;404;403"
Endpoint Object
Each object in the Endpoints
array details how to retrieve the content of the specified endpoint.
Moreover, in each endpoint object, the Metadata
objects lists the content metadata to retrieve.
ClickableUri (String, Required)
The format of the item clickable URI.
Dynamic values are supported.
Example: "ClickableUri": "%[coveo_url]/posts/%[id]"
ItemType (String, Required)
The type of item that the resource contains. Coveo uses this data to organize the indexed content.
Space characters and dashes aren’t supported.
Example: "itemType": "product"
Method (String Enum, Required)
The HTTP method to use to fetch the resource.
Possible values are GET
and POST
.
See the API documentation of the application to make searchable to determine which one you should use.
Example: "Method": "GET"
Path (String, Required)
The relative path to the desired resource endpoint.
Is’s made in the format of /name of the integration object created in the Backoffice/the type of the object
+s
Dynamic values are supported when specified within the SubItems
, SubQueries
, and PermissionSubQueries
objects only.
Dynamic time expressions are also supported.
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, i.e., the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.
Example: "path": "/Product/Products"
Uri (String, Required)
The URI of the product’s PDP page.
Dynamic values are supported.
Example: "Uri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
Should a dynamic value be invalid or unretrievable, the source stops crawling and displays an error in the Administration Console.
Body (String)
The path (simple path or JSONPath) to the Body
content of the item in the JSON response.
The item body is displayed in the Administration Console Content Browser.
Typically, the Body
property contains a dynamic value, since a static, hardcoded value would make all item bodies identical.
Dynamic values and hardcoded text can also be mixed.
Example: "Body": "%[html.content]"
DateFormat (String)
The format of a date injected thanks to the tokens @Now
and @RefreshDate
.
All .NET formats are supported, and default is yyyy-MM-ddTHH:mm:ssZ
(ISO 8601).
Alternatively, to handle Unix timestamps, enter UnixEpoch
.
Examples:
-
"DateFormat": "yyyy-MM-ddTHH:mm:ssZ"
-
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'"
IndexingAction (Action Object)
Whether to ignore or to retrieve an item if the specified Condition
resolves to true.
See the Action Object section for details on the properties expected in the IndexingAction
object.
When the specified Condition
resolves to true, the Coveo crawler applies the specified action to the corresponding item.
Possible actions are Retrieve
and Ignore
, i.e., the crawler can either index the item to make it searchable or ignore it.
An item ignored with IndexingAction
isn’t indexed and therefore not visible in the Administration Console Content Browser or a search interface.
However, its sub-items are sent to the index, unless otherwise specified in the ProcessingAction
object.
If the condition resolves to false, the Coveo crawler falls back to the default action, which is Retrieve
, and the specified item is indexed.
Example:
"IndexingAction": {
"ActionOnItem": "Ignore",
"Condition": "%[id] > 1"
}
IsBinaryBody (Boolean)
Whether the subquery retrieves binary content as the body of the item.
Default value is false
.
This property is available within SubQueries
objects only.
It allows you to download a file (e.g., XLS or PDF document) to use as the item body in a search result quick view.
See Supported File Formats for an exhaustive list of the item types that Coveo can index.
Example:
"SubQueries": [
{
"Headers": {
"x-api-key": "@ApiKey"
},
"Path":"%[coveo_parent.url]",
"Method":"GET",
"IsBinaryBody":true,
"Body": "%[data.body]"
}
]
|
Notes
|
ItemPath (String, Required)
The path to the item in the JSON response returned by the application.
Its value should be d.results
(see
response structure.
Metadata (Object)
A key-value list of the metadata to parse.
This object maps attributes from the integration object to the Coveo fields. Some ecommerce fields are created and mapped by default. You can add customs fields in the Coveo Administration Console (platform-ca | platform-eu | platform-au).
Each key represents the metadata name of the SAP item, while its value is the value path (simple path or JSONPath) in the JSON response.
Typically, the value path consists of one or more dynamic values, since a static, hardcoded value would result in an identical piece of metadata for all items. However, you could choose to use a hardcoded value so that the corresponding Coveo field is filled even if the API doesn’t provide this information.
Example:
"metadata": {
"objecttype": "Product",
"ec_brand": "%[brand]",
"ec_name": "%[name]",
"name": "%[name]",
"ec_item_group_id": "%[code]",
"ec_shortdesc": "%[summary]",
"ec_description": "%[description]",
"ec_price": "%[europe1Prices.results[0].price]",
"ec_thumbnails": "https://accstorefront.commerce.ondemand.com%[thumbnail.URL]",
"ec_in_stock": "%[stock]",
"ec_images": "https://accstorefront.commerce.ondemand.com%[picture.URL]",
"ec_category": "%[supercategories.results[:].name]",
"ec_pricerange": "%[priceRange]",
"ec_configurable": "%[configurable]",
"ec_code": "%[code]",
"ec_url": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
}
ModifiedDate (String)
The date on which the item was last modified. Although this property isn’t required in the JSON configuration, we recommend that you include it, as the Coveo-powered search pages use it for their default Sort by Date option.
Typically, ModifiedDate
has a dynamic value, since a static, hardcoded value would make dates identical for all items.
Example: "ModifiedDate": "%[updated]"
Paging (Paging Object)
In the Paging
object, you can specify how you want the Endpoint
content to be paged.
See the Paging section for details on the properties in this object.
PayloadJsonContent (String)
The JSON content to send as a POST request body.
The Content-Type header is automatically set to application/json
.
Dynamic values are supported when specified within the SubItems
and SubQueries
objects only.
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, i.e., the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.
|
Notes
|
You want to provide the following JSON content with your request:
{
"key1": "value1",
"key2": "@field=(\"value2\")",
"key3":
{
"subkey1": "subvalue1"
}
}
Your source JSON configuration therefore contains the following:
"Endpoints": [
{
"Method": "POST",
"Path": "/rest/myendpoint/",
"PayloadJsonContent": "{\"key1\": \"value1\", \"key2\": \"@field=(\\\"value2\\\")\", \"key3\": {\"subkey1\": \"subvalue1\"}}"
}
]
PayloadParameters (Object)
A key-value list of HTTP parameters to add to the payload. Each value can be either a number, string, or Boolean.
This property can only be used in a POST
request.
The parameters are sent as URL-encoded data in the request body.
The Content-Type header is automatically set to application/x-www-form-urlencoded
.
Dynamic values are supported when specified within the SubItems
and SubQueries
objects only.
Dynamic time expressions are also supported.
Example:
"PayloadParameters": {
"type": "post",
"expand": "true",
"id": 120
}
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, i.e., the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.
|
Note
|
PermanentId (String)
The PermanentId
is an alternative, permanent unique item identifier.
When you expect the item Uri
to change, this ensures that Coveo Machine Learning (Coveo ML) recognizes the items despite their new URI and associates them to the old data.
So, in the future, if you modify or delete this source and reindex the same items, specifying the same item PermanentId
format in your next configuration will ensure that Machine Learning doesn’t consider your items as new and rather picks up where it left off.
Dynamic values are supported.
Example:
{
"Uri": "%[coveo_url]/items/%[sys_id]",
"PermanentId": "%[sys_id]"
}
Permissions (Array of Permission Level Object)
The Permissions
object is required when you select the Same users and groups as in your current permission system source content security option.
It contains the permission levels and sets to apply to the items indexed in this endpoint.
The members listed in these levels and sets are either allowed or denied access to the indexed items.
This property is inheritable.
See the Permission Level Object section for details on the properties in this object.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
ProcessingAction (Action Object)
Whether to ignore or to retrieve the child items of an item if the specified Condition
resolves to true.
See the Action Object section for details on the properties expected in the ProcessingAction
object.
When the specified Condition
resolves to true, the Coveo crawler applies the specified action on the corresponding child items of an item.
Possible actions are Process
and Ignore
, i.e., the crawler can either retrieve the subitems to make them searchable or ignore them.
A child item ignored with ProcessingAction
isn’t indexed and therefore not visible in the Administration Console Content Browser or a search interface.
If the condition resolves to false, it fallbacks to the default action, i.e., Process
.
Example:
"ProcessingAction": {
"ActionOnItem": "Ignore",
"Condition": "%[id]==1"
}
QueryParameters (Object)
A key-value list of HTTP parameters to add to the query. Each value can be either a number, string, or Boolean.
Dynamic values are supported when specified within the SubItems
, SubQueries
, and PermissionSubQueries
objects only.
They aren’t supported in QueryParameters
objects within a RefreshEndpoints
object.
Dynamic time expressions are also supported.
Example:
"QueryParameters": {
"type": "post",
"expand": "true",
"id": 120
}
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, i.e., the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console.
Filter (String)
The expression that specifies the products to be indexed. Must be URL-encoded.
Example: "$filter": "catalogVersion/integrationKey eq 'Online%7CpowertoolsProductCatalog' and approvalStatus/code eq 'approved'"
Expand (String)
The list of entries associated with a single integration object or a collection of integration objects.
Example: "$expand": "supercategories,thumbnails,picture,thumbnail,unit,catalogVersion/catalog,europe1Prices,europe1Prices/currency,bundleTemplates,localizedAttributes,supercategories/localizedAttributes,approvalStatus"
RefreshEndpoints (Array of Refresh Endpoint Object)
A list of refresh endpoint queries to execute to retrieve added, updated, or deleted items when refresh is supported and performed. See the Refresh Endpoints section for details on the properties in this object.
SkippableErrorCodes (String)
When crawling your content, Coveo may encounter an HTTP error and stop.
If you expect the web application API to return an error and want it to be ignored so that the crawling process continues, specify the error code as the SkippableErrorCodes
property value.
This property applies at the Endpoints
level and is inheritable.
Use a semicolon (;
) to separate error codes.
Example: "500;404;403"
SubItems (Array of Endpoint Object)
A list of sub-items to retrieve.
Each sub-item is represented by a configuration object under Endpoints
.
An item can have many different sub-items, e.g., a blog post containing comments and attachments as child items.
The SubItems
object establishes a parent-child relationship between items.
There’s no restriction on the number of sub-items an item can have.
In addition, sub-items can also have sub-items, for example if blog post comments also have attachments.
In the application JSON response, these sub-items are nested within each other, and so should be your SubItems
arrays in your source configuration.
To refer to the metadata of a parent item, prefix the metadata name with coveo_parent
.
You may also need to use raw
.
Examples:
When the sub-items to index are located under a different endpoint, you provide the Path
to their location.
"SubItems": [
{
"Path": "/posts/%[coveo_parent.id]/comments",
"Method": "GET",
"ItemType": "Comment",
"Uri": "%[coveo_url]/repository/posts/%[coveo_parent.id]/comments/%[id]",
"ClickableUri": "%[coveo_url]/posts/%[coveo_parent.id]/comments/%[id]",
"Title": "%[title]",
"ModifiedDate": "%[updated]",
"Body": "%[content]",
"Metadata": {
"author": "%[author]",
"createdby" :"%[coveo_parent.author]",
"id": "%[id]"
}
}
]
Alternatively, when the desired sub-items are located under the same endpoint as their parent items (e.g., the child item data is nested within the parent item data), you omit the Path
property and provide the sub-items' location using the ItemPath
property.
"SubItems": [
{
"ItemPath": "/comments",
"ItemType": "Comment",
"Uri": "%[coveo_url]/repository/posts/%[coveo_parent.id]/comments/%[id]",
"ClickableUri": "%[coveo_url]/posts/%[coveo_parent.id]/comments/%[id]",
"Title": "%[title]",
"ModifiedDate": "%[updated]",
"Body": "%[content]",
"Metadata": {
"author": "%[author]",
"createdby" :"%[coveo_parent.author]",
"id": "%[id]"
}
}
]
SubQueries (Array of Subquery Object)
A list of subqueries to execute on every item in order to fetch additional metadata and/or the item body, especially if this information is located under a different path than that of the item. See the Subquery Object section for details.
To refer to the metadata of a parent item, prefix the metadata name with coveo_parent
.
You may also need to use raw
.
Title (String)
The title of the item. Although this property isn’t required in the JSON configuration, we recommend that you include it, as the Administration Console Content Browser displays its value.
Typically, Title
contains a dynamic value, since a static, hardcoded value would make all item titles identical.
Example: "Title": "%[title]"
Action Object
When the specified Condition
resolves to true, the Coveo crawler applies the specified action to the corresponding items.
See IndexingAction and ProcessingAction for more details.
ActionOnItem (String, Required)
Action to perform if the specified Condition
resolves to true.
In the IndexingAction object, this applies to retrieved items.
Possible actions are Retrieve
and Ignore
, i.e., the crawler can either index the item to make it searchable or ignore it.
In the ProcessingAction object, this applies to child items of an item.
Possible actions are Process
and Ignore
, i.e., the crawler can either retrieve the subitems to make them searchable or ignore them.
Condition (String)
The Condition
value is a condition that must resolve to true for the specified action to apply to the child items of an item.
To write your condition, use the same syntax as for indexing pipeline extension conditions.
Conditions can be assembled together using the following operators: AND
, OR
, Exists
, NOT
, >
, and <
.
Parentheses are also supported to specify operation order.
The metadata fields in your condition must either be defined in the Metadata
object or referenced with raw
.
You can specify an array if the metadata refers to a multi-value field in your repository.
Examples:
-
"%[author_id]"
is true if the item has anauthor_id
. -
"NOT %[author_id]"
is true if the item doesn’t have anauthor_id
. -
"%[author_id] == 1234"
is true if the itemauthor_id
is1234
. -
"%[author_ids] == [1,2,3,4]"
is true if itemauthor_ids
are1
,2
,3
, and4
. -
"%[author_id] OR %[author_name]"
is true if the item has anauthor_id
or anauthor_name
. -
"%[author_id] AND %[author_name]"
is true if the item has anauthor_id
and anauthor_name
. -
"%[author_id] > 123"
is true if the itemauthor_id
is greater than 123. -
"(%[author_id] OR %[author_sys_id]) AND %[author_name]"
is true if the item has anauthor_id
or anauthor_name
, as well as anauthor_name
.
Member Object
A Member
object describes a security identity that can be either allowed or denied in a given permission set.
See AllowedMembers
and DeniedMembers
for more details.
Dynamic values are supported.
You index text items written by different people.
The metadata of each item contains the text author
, and, in this system, author names are email addresses.
With the following configuration, each user in your organization can access the items they have created through your Coveo-powered search interfaces.
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "%[author]",
"Type": "User",
"PermissionType": "config1",
"Condition": "%[field] == value",
"AdditionalInfo": {
"title": "%[customMetadata]"
}
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": false
}
]
Name (String, Required)
The name of the member security identity Email addresses and Active Directory usernames are supported.
Type (String Enum, Required)
The type of member. Allowed values are User, Group, and VirtualGroup.
For more information on these types of security identities, see Coveo Management of Security Identities and Item Permissions.
PermissionType (String, Required)
A key representing the configuration on how to extract all relationships of the security identity specified in Name
.
The security identity provider will use this configuration when processing the security identity.
In the Content Security tab of the Add/Edit an SAP Source panel, you will need to select the Same users and groups as in your current permission system, and then to provide a JSON permission configuration detailing how to retrieve the relationships of each security identity and how to index this data.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
AdditionalInfo (Object)
A key-value store of additional information required by some security providers. Each key represents the metadata name of the repository item, while its value is the value path (simple path or JSONPath) in the JSON response.
When writing a permission configuration, you can use coveo_parent
to refer to the information retrieved with the AdditionalInfo
parameter of a permission subquery.
Permission Condition (String)
The Condition
value is a condition that must resolve to true for the security identity specified in Name
to be resolved, i.e., extracted by the security identity.
As a result, the permission applies.
When the condition resolves to false, the security identity is not resolved and the corresponding permission doesn’t apply.
To write your condition, use the same syntax as for indexing pipeline extension conditions.
Conditions can be assembled together using the following operators: AND
, OR
, Exists
, NOT
, >
, and <
.
Parentheses are also supported to specify operation order.
The metadata fields in your condition must either be defined in the Metadata
object or referenced with raw
.
You can specify an array if the metadata refers to a multi-value field in your repository.
Dynamic values are also supported.
You index a list of available and discontinued products, and your search interface is used by both your customers and your employees.
You want to hide your discontinued products from your customers only, so you use the Condition
parameter to apply a permission to these items.
As a result, all products with Discontinued: yes
in their metadata are visible to members of the *@mycompany.com
group only, i.e.
your employees, while non-discontinued products are available to all search interface users.
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "*@mycompany.com",
"Type": "Group",
"PermissionType": "config1",
"Condition": "%[discontinued] == yes"
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": false
}
]
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
Paging Object
In the Paging
object, specify how you want the content to be paged.
This property is inheritable.
Example:
"Paging": {
"PageSize": 20,
"OffsetStart": 0,
"OffsetType": "page",
"Parameters": {
"Limit":"limit",
"Offset": "page"
}
}
OffsetType (String Enum, Required)
The type of information on which the offset is based.
Possible values are page
, item
, url
, and cursor
.
-
page
: An offset per page is based on the page number, i.e., each paginated call increments the offset by 1.
The offset of the following calls is based on the page number:
-
http://example.com/api/item?limit=10&page=0
-
http://example.com/api/item?limit=10&page=1
-
item
: An offset based on the item number increments the offset by the number of items in each page.
The offset of the following calls is based on the item number:
-
http://example.com/api/item?limit=10&start=0
-
http://example.com/api/item?limit=10&start=10
-
url
: A URL-based offset retrieves the next page (offset) URL with theNextPageKey
in the JSON response, and the next call is a request to this URL.
The following links could be returned in the web application response as values of NextPageKey
.
When you choose to use a URL-based object, Coveo sends a request to this URL to retrieve your content.
-
http://example.com/api/item?limit=5
-
http://example.com/api/item?page=3d170d80d8n3n2342c328s
-
cursor
: An offset based on the cursor retrieves the next page cursor using theNextPageKey
in the JSON response and builds the next page URL.
The offset of the following calls is based on the page cursor:
-
http://example.com/api/item?limit=5
-
http://example.com/api/item?limit=5&pageToken=3d170d80-7b3b-4371-9499-2ec6a78c507e
PageSize (Number, Required)
The number of items to fetch per page.
Example: With paging URL https://example.com/api/item?position=0&quantity=50
, your JSON configuration should include "PageSize": 50
.
NextPageKey (String)
When the OffsetType
value is "url"
or "cursor"
, provide the path (simple path or JSONPath) to the key representing the value of the next page reference, i.e., either the next page URL or the next page cursor.
The NextPageKey
property isn’t required when the OffsetType
value is page
or item
.
Example: "NextPageKey": "NextPageLink"
OffsetStart (Number)
Offset of the first page to fetch.
This property is required when the OffsetType
value is "page"
or "item"
.
Example: Your first paging URL is https://example.com/api/item?position=0&quantity=50
.
If you want to retrieve all items except items 1 to 50, your paging configuration must include "OffsetStart": 1
.
Parameters (Parameter Object)
In the Parameters
object, you must provide the names of the query string fields in your web application pagination URL that are required to support pagination.
The parameters expected in this object are Limit
and Offset
.
However, the Offset
parameter isn’t required when the OffsetType
value is "url"
since the complete URL is already retrieved in the JSON response.
Example:
"Parameters": {
"Limit": "limit",
"Offset": "page"
}
TotalCountHeaderKey (String)
The name of an HTTP header whose value represents the total number of items retrieved through the API call. This property is useful when Coveo makes an API call to a page that doesn’t exist, causing an error to be returned.
Example: "TotalCountHeaderKey": "x-total-count"
TotalCountKey (String)
The path (simple path or JSONPath) to a response body property representing the total number of items retrieved through the API call. This property is useful when Coveo makes an API call to a page that doesn’t exist, causing an error to be returned.
Example: "TotalCountKey": "retrievedContent.totalCount"
Parameter Object
The parameters in the Parameter
object indicate the name of the paging query string fields.
Limit (String, Required)
In the query string, the field whose value is retrieved with the PageSize
parameter.
Example: With paging URL https://example.com/api/item?position=0&quantity=50
, your JSON configuration should include "Limit": "quantity"
.
FetchNextPageUntilNoResult (Boolean)
The FetchNextPageUntilNoResult
property indicates whether Coveo will try to fetch pages until no result is returned, or until the application API returns an error.
Default value is false
.
|
Note
|
Example:
...
"Paging": {
"OffsetType": "page",
"PageSize": 25,
"Parameters": {
"FetchNextPageUntilNoResult": true
}
},
...
Offset (String)
In the query string, the field whose value is retrieved with the OffsetStart
parameter.
Example: Your first paging URL is https://example.com/api/item?position=0&quantity=50
.
If you want to ignore items 1 to 50 and retrieve items 51 and over, your paging configuration must include "Offset": "position"
and "OffsetStart": 1
.
Permission Level Object
A Permission Level object describes a single permission level, which can contain one or more permission sets.
This property is inheritable.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
Example:
"Permissions": [
{
"Name": "Permission Level 1",
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "jsmith@example.com",
"Type": "User",
"AdditionalInfo": {
"title": "%[customMetadata]"
}
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": false
}
]
}
]
Name (String)
The name of the permission level.
PermissionsSets (Array of Permission Set Object)
The PermissionsSets
array specifies who can or can’t access the indexed items.
See the Permission Set Object section for details on the properties expected for each element in this array.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
Permission Set Object
A Permission Set object describes a single permission set which can contain lists of allowed and denied members.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
Permission Set Name (String)
The Name
property indicates the name of the permission set.
AllowedMembers (Array of Member Object)
Each object in the AllowedMembers
array represents a member that should be allowed to access the indexed content.
|
Note
If a user is listed under both For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions. |
See the Member Object section for details on the properties expected in the AllowedMembers
object.
DeniedMembers (Array of Member Object)
Each object in the DeniedMembers
array represents a member to whom access to the indexed item should be denied.
|
Note
If a user is both listed under For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions. |
See the Member Object section for details on the properties expected in the DeniedMembers
object.
IsAnonymousAllowed (Boolean)
The IsAnonymousAllowed
property indicates whether anonymous, i.e., non authenticated users are allowed to access the retrieved items.
Default value is false
.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
PermissionsFromMetadata (Array of PermissionsFromMetadata Object)
The PermissionsFromMetadata
object is an alternative to the PermissionSubqueries
object, as it fetches security identity relationship data, typically the security identities that are children of a group.
See the PermissionsFromMetadata Object section for details on the properties expected for each element in this array.
PermissionSubQueries (Array of Permission Subquery Object)
The PermissionSubQueries
array contains a list of subqueries to execute on every item to fetch permission data, typically the security identities that are allowed or denied access to the item.
This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo.
The PermissionSubQueries
array si an alternative to the PermissionsFromMetadata
object.
See the Permission Subquery Object section for details on the items found in an array.
PermissionsFromMetadata Object
The PermissionsFromMetadata
object is an alternative to the PermissionSubQueries
object, as it fetches permission data, typically the security identities that are allowed or denied access to the item.
This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo.
Include a PermissionsFromMetadata
object in your SAP source configuration when the permission data appears in the metadata of an item.
Conversely, a PermissionSubQueries
object should be used when an item and its permission data must be retrieved with two different queries.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
The PermissionsFromMetadata
object contains:
-
A subset of the fields from the
Member
object configuration: -
The following property:
Permission Subquery Object
The PermissionSubQueries
object is a list of subqueries to execute on every item in order to fetch permission data, typically the security identities that are allowed or denied access to the item.
This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo.
It is an alternative to the PermissionsFromMetadata
object.
Include a PermissionSubQueries
object in your SAP source configuration when an item and its permission data must be retrieved with two different queries. Conversely, the PermissionsFromMetadata
object should be used when the permission data appears in the metadata of an item.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
The subquery configuration contains:
Example:
"permissionSubQueries": [
{
"ItemPath": "permissions",
"Path": "%[productid]/permissions",
"Method": "GET",
"QueryParameters": {},
"Name": "%[name]",
"Type": "group",
"PermissionType": "config1",
"Condition": "%[field] == value",
"IsAllowedMember": true,
"AdditionalInfo":
{
"PermissionID": "%[id]",
"PermissionName": "%[name]"
}
}
]
IsAllowedMember (Boolean)
The IsAllowedMember
property indicates whether the specified member is allowed to access the item.
Default value is false
.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo Management of Security Identities and Item Permissions.
Refresh Endpoint Object
A refresh endpoint is an endpoint that inherits from the parent endpoint definition.
The refresh endpoint allows you to redefine the Endpoints configuration properties.
As a result, when you refresh your source, the properties defined under RefreshEndpoints
override the properties defined under Endpoints
.
When you rescan or rebuild the source, however, the Endpoints
properties apply.
In addition to the Endpoints
object configuration properties, the RefreshEndpoints
object supports the following properties:
|
Note
Dynamic values aren’t supported in the query parameters of a |
Example:
"RefreshEndpoints":[
{
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters":{
"workflow_state":"published",
"since": "@RefreshDate"
}
},
{
"IsDeletionQuery":true,
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters":{
"lastModifiedDate": "@Now-7d",
"status":"Deleted"
}
},
{
"IsDeletionQuery":true,
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"Path":"/api/table/audit_delete",
"QueryParameters":{
"tablename":"kb_knowledge",
"since": "@RefreshDate"
}
},
{
"IsDeletedItem": "%[is_deleted]",
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters":{
"since": "@RefreshDate"
}
}
]
IsDeletionQuery (Boolean)
Whether the defined query returns items to delete from the index.
Default value is false
, which means that all retrieved items are added to the index or updated.
If the query returns items to add, update, and delete all at once, use the IsDeletedItem
property instead.
IsDeletedItem (Boolean)
When a refresh query returns items to add, update, and delete all at once, this property allows you to specify which of these items should be deleted.
Default value is false
, but it’s considered to be true
if IsDeletionQuery
is set to true
.
You can provide a dynamic value (e.g., %[has_been_deleted_flag]
) to be evaluated for each item or a static value.
Example:
"%[is_deleted]"
deletes all items that have a "is_deleted": true
JSON property flagging them as deleted.
"IsDeletedItem": %[is_deleted],
"DateFormat":"\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters":{
"since": "@RefreshDate"
}
DeleteChildren (Boolean)
Whether to delete children when the retrieved parent item is deleted.
Default value is false
.
If the value is set to true
, child items are deleted based on their Uri
.
Subquery Object
A list of subqueries to execute on every item in order to fetch additional metadata and/or the item body, especially if this information is located under a different path than that of the item.
The subquery configuration contains a subset of the fields from the Endpoints
object configuration:
It also contains the following parameters:
Example:
"SubQueries": [
{
"Path": "/posts/%[coveo_parent.id]/additionalinfo",
"Method": "GET",
"Body": "%[data.body]",
"Metadata": {
"created": "%[created]"
}
}
]
IsThumbnail (Boolean)
If the SubQueries
object path leads to an image, set IsThumbnail
to true
to show this image as the thumbnail of an item.
The default value is false
.
Supported thumbnail file formats are the following:
.bmp
, .emf
, .exif
, .gif
, .icon
, .jpeg
, .png
, .tiff
, .wmf
.
|
Notes
|
Authentication Object
If one of the Services to crawl requires authentication, your JSON configuration must include the following properties.
Username (String)
Username used to log in to the REST service.
You can use @Username
to retrieve the value specified in the Add an SAP Source panel.
Password (String)
Password used to log in to the REST service.
This property can be empty even though a Username
is specified.
You can use @Password
to retrieve the value specified in the Add an SAP Source panel.
Domain (String)
Domain name to use when authenticating to a NTLM or Kerberos protected server.
ForceBasicAuthentication (Boolean)
Whether to force a basic HTTP header in the request.
The default value is false
.
OAuth (OAuth Object)
If the application to make searchable requires OAuth 2.0 authentication, the OAuth
object must contain the required configuration.
You can use the @
syntax to map the credentials or the API key you specified in the Add an SAP Source panel.
See the OAuth 2.0 section for details on the object properties.
Example:
"OAuth": {
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "REFRESH_TOKEN",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "CLIENT_ID"
},
"client_secret": {
"Type": "Payload",
"Value": "CLIENT_SECRET"
}
}
},
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
}
OAuth Object
If the application to make searchable requires OAuth 2.0 authentication, include the following properties in your source JSON configuration.
Query (OAuth Query Object, Required)
The Query
object represents the OAuth 2.0 refresh query configuration.
See OAuth 2.0 Query for the properties expected in this object.
Enter your credentials in the Add an SAP Source panel and use the @ClientId
, @ClientSecret
, and @RefreshToken
placeholders in your JSON configuration.
Response (OAuth Response Object)
The Response
object represents the OAuth 2.0 refresh query configuration.
See OAuth 2.0 Response for the properties expected in this object.
OAuth Query Object
If the application to make searchable requires OAuth 2.0 authentication, include the following properties in your JSON configuration.
Example:
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}
RefreshUrl (String, Required)
The OAuth 2.0 refresh URL.
Example: "RefreshUrl": "http://example.com/token"
Authentication Headers (Object)
The additional HTTP headers to use to authenticate the client and refresh the token.
|
Note
Coveo automatically sets the Content-Type header. Therefore, you don’t need to enter anything for this header. |
Example:
"Headers": {
"X-Auth": "API_KEY"
}
Method (String Enum)
The HTTP method to use to refresh the OAuth 2.0 token.
Default is POST
.
Parameters (Object)
The parameters to use to authenticate the client and refresh the token.
Possible values are query
or payload
.
If the OAuth 2.0 flow to use doesn’t support refresh tokens, don’t specify it in the Query
object parameters.
Enter your credentials in the Add an SAP Source panel and use the @ClientId
, @ClientSecret
, and @RefreshToken
placeholders in your JSON configuration.
Example:
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
OAuth Response Object
If the application to make searchable requires OAuth 2.0 authentication, include the following properties in your JSON configuration.
Example:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
AccessToken (String)
The path to the access token in the response.
Default is access_token
.
SupportsRefreshToken (Boolean)
Whether the OAuth 2.0 flow supports refresh tokens.
Default is true
.
RefreshToken (String)
The path to the refresh token in the response.
Default is refresh_token
.
ExpiresIn (String)
The path to the expiration time in the response.
Default is expires_in
.
If there’s no expiration time in the response, use the ExpiresInDefaultValue
property.
ExpiresInDefaultValue (Number)
Specify an expiration value in seconds for the refresh token as an alternative to the ExpiresIn
property.
Default is 3600
.
TokenType (String)
The path to the token type in the response.
Default is token_type
.
If the TokenType
isn’t specified in the response or if it’s invalid or inappropriate, it defaults to Bearer
.
If you want a different default value, use the TokenTypeDefaultValue
property.
TokenTypeDefaultValue (String)
If the TokenType
isn’t specified in the response or if it’s invalid or inappropriate, it defaults to Bearer
.
Should you not want to use Bearer
as a replacement for the inadequate token_type
value that the response returns, specify the desired value.
AuthorizationHeader (String)
Specify the name of the authorization header where the access token should appear.
Default is Authorization
.
Example:
"AuthorizationHeader": "X-ACCESS-TOKEN"