SAP source reference
SAP source reference
When creating an SAP source in the Coveo Administration Console, you must provide a JSON configuration detailing which services and endpoints to crawl to retrieve the desired content, and how to retrieve items of each type.
This configuration has a single property, Services, whose value must be an array of objects.
{
"services": [
{
"url": "https://api.acmestore.com/odata2webservices",
"headers": {
"Accept": "application/json",
"Application-Interface-Key": "xxxxxxxx"
},
"paging": {
"pageSize": 10,
"offsetStart": 0,
"offsetType": "item",
"nextPageKey": "__next",
"parameters": {
"limit": "$top",
"offset": "$skip"
}
},
"endpoints": [
{
"path": "/Product/Products",
"method": "GET",
"queryParameters": {
"$filter": "catalogVersion/integrationKey eq 'Online%7CpowertoolsProductCatalog' and approvalStatus/code eq 'approved'",
"$expand": "supercategories,thumbnails,picture,thumbnail,unit,catalogVersion/catalog,europe1Prices,europe1Prices/currency,bundleTemplates,localizedAttributes,supercategories/localizedAttributes,approvalStatus"
},
"itemPath": "d.results",
"itemType": "product",
"uri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]",
"clickableUri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]",
"metadata": {
"objecttype": "Product",
"ec_brand": "%[brand]",
"ec_name": "%[name]",
"name": "%[name]",
"ec_item_group_id": "%[code]",
"ec_shortdesc": "%[summary]",
"ec_description": "%[description]",
"ec_price": "%[europe1Prices.results[0].price]",
"ec_thumbnails": "https://accstorefront.commerce.ondemand.com%[thumbnail.URL]",
"ec_in_stock": "%[stock]",
"ec_images": "https://accstorefront.commerce.ondemand.com%[picture.URL]",
"ec_category": "%[supercategories.results[:].name]",
"ec_pricerange": "%[priceRange]",
"ec_configurable": "%[configurable]",
"ec_code": "%[code]",
"ec_url": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
}
}
]
}
]
}This reference article defines parameters to include in your JSON configuration.
When working on your SAP source, you may also want to refer to the following articles:
|
Use online tools such as JSONPath Online Evaluator to test your JSON paths, and freeformatter.com to validate and escape your JSON configuration. |
Table of contents
To help you jump to the right section, here’s a hierarchical and alphabetical list of all supported arrays, objects, and parameters.
Show table of contents
| Hierarchical list | Alphabetical list |
|---|---|
|
|
Services (array, required)
Each object in the Services array specifies an SAP API service to retrieve content, along with the specific endpoints to call.
The properties that may appear in an object of the Services array are the following:
Endpoints (array, required)
Services > Endpoints
Each object in the Endpoints array details how you want Coveo to crawl an endpoint of your SAP Commerce instance.
Moreover, in each endpoint object, the Metadata objects lists the content metadata to retrieve.
By default, the SAP source JSON configuration contains three endpoints, one for:
-
product (for example,
"path": "/Product/Products") -
variant (for example,
"path": "FootWearVariant/GenericVariantProducts") -
availability (for example,
"path": "/Availability/Warehouses")
You can remove or add endpoints depending on your use case.
Objects of the Endpoints array support the following properties from their parent service object:
It also supports the following properties:
-
ClickableUri(required) -
ItemPath(required) -
ItemType(required) -
Path(required) -
Uri(required)
ClickableUri (string, required)
The ClickableUri is the URI of the page to open when a user clicks the title of an item from your SAP source in a Coveo-powered search interface.
It can be identical to the Uri.
Dynamic values are supported.
Example: "ClickableUri": "%[coveo_url]/posts/%[id]"
ItemType (string, required)
The type of item that the resource contains. Coveo uses this data to organize the indexed content.
Space characters and dashes aren’t supported.
Example: "itemType": "product"
Path (string, required)
The relative path to the desired resource endpoint.
It’s made in the format of /name of the integration object created in the Backoffice/the type of the object+s.
A Path string cannot contain dynamic values if it’s located directly in an Endpoint or RefreshEndpoint object.
Since dynamic values are placeholders for data that Coveo extracts from your API response, an Endpoint or RefreshEndpoint query that contains a dynamic value is invalid.
Because it hasn’t made the query yet, Coveo doesn’t have an API response to use to resolve the dynamic value.
On the other hand, dynamic values are supported in a Path string located within SubItems, SubQueries, and PermissionSubQueries arrays.
Since these arrays represent queries to make after the main Endpoint or RefreshEndpoint query, Coveo will be able to resolve the dynamic values using the API response from the main query and the item metadata (as well as the parent item’s metadata, if applicable).
Dynamic time expressions are supported.
Example: "path": "/Product/Products"
|
Note
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console. |
Uri (string, required)
The URI of the product’s PDP page.
Dynamic values are supported.
Example: "Uri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
Should a dynamic value be invalid or unretrievable, the source stops crawling and displays an error in the Administration Console.
|
This value populates the unique identifier of the source item in the index.
If the computed value of |
Body (string)
The path (simple path or JSONPath) to the Body content of the item in the JSON response.
The item body is displayed in the Content Browser.
Typically, the Body property contains a dynamic value, since a static, hardcoded value would make all item bodies identical.
Dynamic values and hardcoded text can also be mixed.
Example: "Body": "%[html.content]"
DateFormat (string)
The format of a date injected thanks to the tokens @Now and @RefreshDate.
All .NET formats are supported, and the default is yyyy-MM-ddTHH:mm:ssZ (ISO 8601).
Alternatively, to handle Unix timestamps, enter UnixEpoch.
Coveo also uses this format when parsing dates coming from your API.
If these dates aren’t in a .NET format, use DateFormat to specify the format to expect.
Examples:
-
"DateFormat": "yyyy-MM-ddTHH:mm:ssZ" -
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'"
ExtendedMetadata (object)
In the ExtendedMetadata object, you can add rules to alter the metadata that Coveo indexes with a dynamic value.
Each rule is defined by a key-value pair where the key is the name of the rule and the value is an object that contains the rule configuration.
Conversely, your Metadata object should define the metadata that you want to index as it is returned by the API.
For example, your source JSON configuration could contain the following:
"ExtendedMetadata": {
"ec_categories": { 
}
},
"Metadata": {
"author": "%[author_fullname]", 
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
In the author field, Coveo will index the string that your API returns under author_fullname. |
Each rule object must contain an Expression object with a Value property, and either of the following:
-
The
IsDynamicConditionBoolean property set totruein theExpressionobject.Example: Changing a number to a Boolean
The following rule evaluates whether the value retrieved with the
%[inventoryQuantity]dynamic value is greater than 0. The result is indexed in thein_inventoryfield as a Boolean value. So, if your API returns"inventoryQuantity": "409", Coveo indexestrue."ExtendedMetadata": { "in_inventory": {
"Expression": {
"Value": "%[inventoryQuantity] > 0",
"IsDynamicCondition": true 
}
}
}In the in_inventoryfield, Coveo will index the result of the rule in this object.%[inventoryQuantity]is a dynamic value that retrieves the metadata stored underinventoryQuantityin your API response. With thisValueproperty, Coveo evaluates whether the retrieved value is greater than 0.The IsDynamicConditionproperty is set totrueto indicate that theValueproperty is a dynamic condition to evaluate. -
The
ExtendedMetadataActionsarray, which contains one or more objects that define the actions to apply to the metadata.Example: Replacing a character in a string
The following rule replaces
>with a;character in values retrieved with the%[category]dynamic value. So, if your API returns"category": "Electronics > Phones", Coveo indexes"Electronics;Phones"in theec_categoriesfield."ExtendedMetadata": { "ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": " > ",
"NewValue": ";"
},
{
"ActionName": "Default", 
"DefaultValue": "No category"
}
]
}
}In the ec_categoriesfield, Coveo will index the result of the rule in this object.%[category]is a dynamic value that retrieves the metadata stored undercategoryin your API response.Coveo will alter the metadata retrieved with %[category]according to the rules specified in theExtendedMetadataActionsarray.If Coveo can’t find the OldValuein the metadata, it moves on to the next rule. With the second rule, if no category metadata is returned, Coveo indexesUnknown categoryin thecategoriesfield.Then, if you select the Multi-value facet option for the
ec_categoriesfield, your search interface will allow users to filter results by selectingElectronicsand/orPhones.
Example: Using the
Replaceaction to trim part of a stringLet’s say you want to index product IDs in the
product_idfield. If the API that you call returns ID values such asgid://Store/Product/12345, you need to trim thegid://Store/Product/part to only index the actual product ID.The following rule replaces
gid://Store/Product/with an empty string in values retrieved with the%[id]dynamic value. So, if your API returns"id": "gid://Store/Product/12345", Coveo indexes12345in theproduct_idfield."ExtendedMetadata": { "product_id": {
"Expression": {
"value": "%[id]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": "gid://shopify/Product/",
"NewValue": ""
}
]
}
}In the product_idfield, Coveo will index the result of the rule in this object.%[id]is a dynamic value that retrieves the metadata stored underidin your API response.Coveo will alter the metadata retrieved with %[id]according to the rules specified in theExtendedMetadataActionsarray.Example: Replacing a missing value with a default value
The following rule replaces a missing metadata value with a default value. So, if your API returns
"name_author": ""or"name_author": null, Coveo indexes"Unknown author"in theauthorfield."ExtendedMetadata": { "author": {
"Expression": {
"Value": "%[name_author]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "Unknown author",
}
]
}
}In the authorfield, Coveo will index the result of the rule in this object.%[name_author]is a dynamic value that retrieves the metadata stored undername_authorin your API response.Coveo will alter the metadata retrieved with the %[name_author]according to the rules specified in theExtendedMetadataActionsarray.Example: Replacing a missing translation with a default value
Let’s say you want to index product title translations in the
product_titlefield. However, if a translation is missing, you want to index the product title in the default language instead.The following rule retrieves the
titlevalue from thelocaleTranslationsobject in your API response. If the translation is missing, Coveo indexes the product title in the default language in theproduct_titlefield."ExtendedMetadata": { "product_title": {
"Expression": {
"value": "%[localeTranslations[?(@.key == 'title')].value]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "%[title]"
}
]
}
}In the product_titlefield, Coveo will index the result of the rule in this object.This query retrieves the metadata stored under localeTranslationsin your API response and filters it to only return thetitlevalue.Coveo will alter the metadata retrieved with the query according to the rules specified in the ExtendedMetadataActionsarray.
Expression (object, required)
Services > Endpoints > ExtendedMetadata > Expression
When configuring your source, you may add rules instructing your source to alter the metadata it’s about to index.
When you add such a rule in the ExtendedMetadata object, you must use the Expression object to specify, among other things, the metadata targeted by the rule.
The Expression object supports the following properties:
-
Value: The condition to evaluate or the metadata to alter.Valueis required when adding anExpressionobject. -
IsDynamicCondition: Whether theValueproperty is a dynamic condition.
Example 1: Changing a number to a Boolean
The following rule evaluates whether the value retrieved with the %[inventoryQuantity] dynamic value is greater than 0.
The result is indexed in the in_inventory field as a Boolean value.
So, if your API returns "inventoryQuantity": "409", Coveo indexes true.
"ExtendedMetadata": {
"in_inventory": {
"Expression": {
"Value": "%[inventoryQuantity] > 0",
"IsDynamicCondition": true
}
}
} |
In the in_inventory field, Coveo will index the result of the rule in this object. |
|
%[inventoryQuantity] is a dynamic value that retrieves the metadata stored under inventoryQuantity in your API response.
With this Value property, Coveo evaluates whether the retrieved value is greater than 0. |
|
The IsDynamicCondition property is set to true to indicate that the Value property is a dynamic condition to evaluate. |
Example 2: Replacing a character in a string
The following rule replaces > with a ; character in values retrieved with the %[category] dynamic value.
So, if your API returns "category": "Electronics > Phones", Coveo indexes "Electronics;Phones" in the ec_categories field.
"ExtendedMetadata": {
"ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": " > ",
"NewValue": ";"
},
{
"ActionName": "Default",
"DefaultValue": "No category"
}
]
}
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
%[category] is a dynamic value that retrieves the metadata stored under category in your API response. |
|
Coveo will alter the metadata retrieved with %[category] according to the rules specified in the ExtendedMetadataActions array. |
|
If Coveo can’t find the OldValue in the metadata, it moves on to the next rule.
With the second rule, if no category metadata is returned, Coveo indexes Unknown category in the categories field. |
Then, if you select the Multi-value facet option for the ec_categories field, your search interface will allow users to filter results by selecting Electronics and/or Phones.
Value (string, required)
Services > Endpoints > ExtendedMetadata > Expression > Value
When configuring your source, you may add rules instructing your source to alter the metadata it’s about to index.
In such case, use the Value property to specify what the ExtendedMetadata rule targets.
The Value property can be either of the following:
-
A dynamic value representing the piece of metadata to alter.
Example: Replacing a character in a string
The following rule replaces
>with a;character in values retrieved with the%[category]dynamic value. So, if your API returns"category": "Electronics > Phones", Coveo indexes"Electronics;Phones"in theec_categoriesfield."ExtendedMetadata": { "ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": " > ",
"NewValue": ";"
},
{
"ActionName": "Default",
"DefaultValue": "No category"
}
]
}
}In the ec_categoriesfield, Coveo will index the result of the rule in this object.%[category]is a dynamic value that retrieves the metadata stored undercategoryin your API response.Coveo will alter the metadata retrieved with %[category]according to the rules specified in theExtendedMetadataActionsarray.If Coveo can’t find the OldValuein the metadata, it moves on to the next rule. With the second rule, if no category metadata is returned, Coveo indexesUnknown categoryin thecategoriesfield.Then, if you select the Multi-value facet option for the
ec_categoriesfield, your search interface will allow users to filter results by selectingElectronicsand/orPhones.
Example: Replacing a missing value with a default value
The following rule replaces a missing metadata value with a default value. So, if your API returns
"name_author": ""or"name_author": null, Coveo indexes"Unknown author"in theauthorfield."ExtendedMetadata": { "author": {
"Expression": {
"Value": "%[name_author]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "Unknown author",
}
]
}
}In the authorfield, Coveo will index the result of the rule in this object.%[name_author]is a dynamic value that retrieves the metadata stored undername_authorin your API response.Coveo will alter the metadata retrieved with the %[name_author]according to the rules specified in theExtendedMetadataActionsarray.Example: Filtering the metadata to return a single value
Let’s say you want to index product title translations in the
product_titlefield. However, if a translation is missing, you want to index the product title in the default language instead.The following rule retrieves the
titlevalue from thelocaleTranslationsobject in your API response. If the translation is missing, Coveo indexes the product title in the default language in theproduct_titlefield."ExtendedMetadata": { "product_title": {
"Expression": {
"value": "%[localeTranslations[?(@.key == 'title')].value]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "%[title]"
}
]
}
}In the product_titlefield, Coveo will index the result of the rule in this object.This query retrieves the metadata stored under localeTranslationsin your API response and filters it to only return thetitlevalue.Coveo will alter the metadata retrieved with the query according to the rules specified in the ExtendedMetadataActionsarray. -
An expression to evaluate. This expression must contain a dynamic value.
Example: Changing a number to a Boolean
The following rule evaluates whether the value retrieved with the
%[inventoryQuantity]dynamic value is greater than 0. The result is indexed in thein_inventoryfield as a Boolean value. So, if your API returns"inventoryQuantity": "409", Coveo indexestrue."ExtendedMetadata": { "in_inventory": {
"Expression": {
"Value": "%[inventoryQuantity] > 0",
"IsDynamicCondition": true
}
}
}In the in_inventoryfield, Coveo will index the result of the rule in this object.%[inventoryQuantity]is a dynamic value that retrieves the metadata stored underinventoryQuantityin your API response. With thisValueproperty, Coveo evaluates whether the retrieved value is greater than 0.The IsDynamicConditionproperty is set totrueto indicate that theValueproperty is a dynamic condition to evaluate.
When using Value to provide an expression to evaluate, the syntax to use is the same as for indexing pipeline extension conditions.
In such case, you must also make sure to add the IsDynamicCondition property to your Expression object.
Conditions can be assembled using the following operators: AND, OR, Exists, NOT, >, and <.
Parentheses are also supported to specify operation order.
Examples
-
"%[author_id]"is true if the item has anauthor_id. -
"NOT %[author_id]"is true if the item doesn’t have anauthor_id. -
"%[author_id] == 1234"is true if the itemauthor_idis1234. -
"%[author_ids] == [1,2,3,4]"is true if itemauthor_idsare1,2,3, and4. -
"%[author_id] OR %[author_name]"is true if the item has anauthor_idor anauthor_name. -
"%[author_id] AND %[author_name]"is true if the item has anauthor_idand anauthor_name. -
"%[author_id] > 123"is true if the itemauthor_idis greater than 123. -
"(%[author_id] OR %[author_sys_id]) AND %[author_name]"is true if the item has anauthor_idor anauthor_name, as well as anauthor_name.
IsDynamicCondition (Boolean)
Services > Endpoints > ExtendedMetadata > Expression > IsDynamicCondition
When configuring your source, you may add rules instructing your source to alter the metadata it’s about to index.
In such case, use the IsDynamicCondition property to specify whether the Value property in the Expression object is a condition to evaluate.
The default value is false.
When you set IsDynamicCondition to true, Coveo evaluates the expression and indexes the Boolean result in the field specified under Metadata.
Example
The following rule evaluates whether the value retrieved with the %[inventoryQuantity] dynamic value is greater than 0.
The result is indexed in the in_inventory field as a Boolean value.
So, if your API returns "inventoryQuantity": "409", Coveo indexes true.
"ExtendedMetadata": {
"in_inventory": {
"Expression": {
"Value": "%[inventoryQuantity] > 0",
"IsDynamicCondition": true
}
}
} |
In the in_inventory field, Coveo will index the result of the rule in this object. |
|
%[inventoryQuantity] is a dynamic value that retrieves the metadata stored under inventoryQuantity in your API response.
With this Value property, Coveo evaluates whether the retrieved value is greater than 0. |
|
The IsDynamicCondition property is set to true to indicate that the Value property is a dynamic condition to evaluate. |
ExtendedMetadataActions (array)
Services > Endpoints > ExtendedMetadata > ExtendedMetadataActions
When configuring your source, you may add rules instructing your source to alter the metadata it’s about to index.
In such case, you may add an ExtendedMetadataActions array to your rule to specify the actions to perform on the targeted metadata.
Each object in the array represents an action to perform on the metadata.
The ExtendedMetadataActions array is needed when the Value property contains a single dynamic value representing the metadata to alter.
Example 1
The following rule replaces > with a ; character in values retrieved with the %[category] dynamic value.
So, if your API returns "category": "Electronics > Phones", Coveo indexes "Electronics;Phones" in the ec_categories field.
"ExtendedMetadata": {
"ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": " > ",
"NewValue": ";"
},
{
"ActionName": "Default",
"DefaultValue": "No category"
}
]
}
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
%[category] is a dynamic value that retrieves the metadata stored under category in your API response. |
|
Coveo will alter the metadata retrieved with %[category] according to the rules specified in the ExtendedMetadataActions array. |
|
If Coveo can’t find the OldValue in the metadata, it moves on to the next rule.
With the second rule, if no category metadata is returned, Coveo indexes Unknown category in the categories field. |
Then, if you select the Multi-value facet option for the ec_categories field, your search interface will allow users to filter results by selecting Electronics and/or Phones.
When the array contains multiple objects, Coveo tries applying the first action. If the action cannot be performed, Coveo tries the next action, and so on.
Example 2
The following rule replaces > with a ; character in values retrieved with the %[category] dynamic value.
So, if your API returns "category": "Electronics > Phones", Coveo indexes "Electronics;Phones" in the ec_categories field.
"ExtendedMetadata": {
"ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": " > ",
"NewValue": ";"
},
{
"ActionName": "Default",
"DefaultValue": "No category"
}
]
}
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
%[category] is a dynamic value that retrieves the metadata stored under category in your API response. |
|
Coveo will alter the metadata retrieved with %[category] according to the rules specified in the ExtendedMetadataActions array. |
|
If Coveo can’t find the OldValue in the metadata, it moves on to the next rule.
With the second rule, if no category metadata is returned, Coveo indexes Unknown category in the categories field. |
Then, if you select the Multi-value facet option for the ec_categories field, your search interface will allow users to filter results by selecting Electronics and/or Phones.
Each ExtendedMetadataActions object supports the following properties:
-
ActionName: The name of the action to perform on the metadata targeted by the rule. This property is required in each object of theExtendedMetadataActionsarray. -
OldValue: What to replace in the metadata. -
NewValue: What to replace theOldValuewith. -
DefaultValue: The default value to use if the metadata targeted by the rule is not found. -
FallbackValuesTrigger: What to replace withDefaultValue. -
StringSeparator: The character or string that separates the hierarchical levels in the metadata value returned by your API.
ActionName (string enum, required)
Services > Endpoints > ExtendedMetadata > ExtendedMetadataActions > ActionName
When configuring your source, you may add rules instructing your source to alter the metadata it’s about to index.
In such case, you can use the ActionName property to specify the action to perform on the metadata retrieved with the dynamic value specified under Value.
Possible values are the following:
ActionName value |
Action description | Properties to add to the ExtendedMetadataActions object
|
|---|---|---|
|
Replaces a character or string in the metadata.
The content of the |
|
|
Replaces missing metadata with a default value.
So, if your API returns |
|
|
Formats the metadata so that it can be displayed in a hierarchical facet. |
|
|
Removes leading and trailing whitespace characters from the metadata. |
None |
Example: Replacing a character in a string
The following rule replaces > with a ; character in values retrieved with the %[category] dynamic value.
So, if your API returns "category": "Electronics > Phones", Coveo indexes "Electronics;Phones" in the ec_categories field.
"ExtendedMetadata": {
"ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": " > ",
"NewValue": ";"
},
{
"ActionName": "Default",
"DefaultValue": "No category"
}
]
}
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
%[category] is a dynamic value that retrieves the metadata stored under category in your API response. |
|
Coveo will alter the metadata retrieved with %[category] according to the rules specified in the ExtendedMetadataActions array. |
|
If Coveo can’t find the OldValue in the metadata, it moves on to the next rule.
With the second rule, if no category metadata is returned, Coveo indexes Unknown category in the categories field. |
Then, if you select the Multi-value facet option for the ec_categories field, your search interface will allow users to filter results by selecting Electronics and/or Phones.
Example: Replacing a missing value with a default value
The following rule replaces a missing metadata value with a default value.
So, if your API returns "name_author": "" or "name_author": null, Coveo indexes "Unknown author" in the author field.
"ExtendedMetadata": {
"author": {
"Expression": {
"Value": "%[name_author]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "Unknown author",
}
]
}
} |
In the author field, Coveo will index the result of the rule in this object. |
|
%[name_author] is a dynamic value that retrieves the metadata stored under name_author in your API response. |
|
Coveo will alter the metadata retrieved with the %[name_author] according to the rules specified in the ExtendedMetadataActions array. |
Example: Using the Replace action to trim part of a string
Let’s say you want to index product IDs in the product_id field.
If the API that you call returns ID values such as gid://Store/Product/12345, you need to trim the gid://Store/Product/ part to only index the actual product ID.
The following rule replaces gid://Store/Product/ with an empty string in values retrieved with the %[id] dynamic value.
So, if your API returns "id": "gid://Store/Product/12345", Coveo indexes 12345 in the product_id field.
"ExtendedMetadata": {
"product_id": {
"Expression": {
"value": "%[id]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": "gid://shopify/Product/",
"NewValue": ""
}
]
}
} |
In the product_id field, Coveo will index the result of the rule in this object. |
|
%[id] is a dynamic value that retrieves the metadata stored under id in your API response. |
|
Coveo will alter the metadata retrieved with %[id] according to the rules specified in the ExtendedMetadataActions array. |
Example: Formatting metadata as a hierarchy
You use the FormatHierarchical action because you’re planning to add a hierarchical facet to your Coveo-powered search interface.
This action reformats your metadata in the way that’s required for the hierarchical facet to work properly.
Thanks to the StringSeparator property, Coveo can identify the hierarchy levels in the metadata.
So, if your API returns "category": "Men > Clothing > Shoes", Coveo indexes Men;Men|Clothing;Men|Clothing|Shoes in the ec_categories field.
Then, once you add a hierarchical facet based on the ec_categories field to your search interface, your search interface will allow users to filter results by selecting a category such as Men, and then a subcategory such as Shoes, and so on.
"ExtendedMetadata": {
"ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "FormatHierarchical",
"StringSeparator": " > "
}
]
}
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
%[category] is a dynamic value that retrieves the metadata stored under category in your API response. |
|
Coveo will alter the metadata retrieved with %[category] according to the rules specified in the ExtendedMetadataActions array. |
|
Thanks to the StringSeparator property, Coveo can identify the hierarchy levels in the metadata. |
Example: Trimming leading and trailing whitespace characters
The following rule trims leading and trailing whitespace characters from values retrieved with the %[category] dynamic value.
So, if your API returns "category": " Surf boards ", Coveo indexes Surf boards in the ec_categories field.
"ExtendedMetadata": {
"ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Trim"
}
]
}
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
%[category] is a dynamic value that retrieves the metadata stored under category in your API response. |
|
Coveo will alter the metadata retrieved with %[category] according to the rules specified in the ExtendedMetadataActions array. |
Example: Filtering the metadata to return a single value
Let’s say you want to index product title translations in the product_title field.
However, if a translation is missing, you want to index the product title in the default language instead.
The following rule retrieves the title value from the localeTranslations object in your API response.
If the translation is missing, Coveo indexes the product title in the default language in the product_title field.
"ExtendedMetadata": {
"product_title": {
"Expression": {
"value": "%[localeTranslations[?(@.key == 'title')].value]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "%[title]"
}
]
}
} |
In the product_title field, Coveo will index the result of the rule in this object. |
|
This query retrieves the metadata stored under localeTranslations in your API response and filters it to only return the title value. |
|
Coveo will alter the metadata retrieved with the query according to the rules specified in the ExtendedMetadataActions array. |
OldValue and NewValue (string)
Services > Endpoints > ExtendedMetadata > ExtendedMetadataActions > OldValue/NewValue
When configuring your source, you may add rules instructing your source to alter the metadata it’s about to index.
In such case, if your rule object contains a Replace action, it must also include the following properties:
-
OldValue, where you specify the part of the string to replace. -
NewValue, where you specify what this part should be replaced with.
Example: Replacing a character in a string
The following rule replaces > with a ; character in values retrieved with the %[category] dynamic value.
So, if your API returns "category": "Electronics > Phones", Coveo indexes "Electronics;Phones" in the ec_categories field.
"ExtendedMetadata": {
"ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": " > ",
"NewValue": ";"
},
{
"ActionName": "Default",
"DefaultValue": "No category"
}
]
}
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
%[category] is a dynamic value that retrieves the metadata stored under category in your API response. |
|
Coveo will alter the metadata retrieved with %[category] according to the rules specified in the ExtendedMetadataActions array. |
|
If Coveo can’t find the OldValue in the metadata, it moves on to the next rule.
With the second rule, if no category metadata is returned, Coveo indexes Unknown category in the categories field. |
Then, if you select the Multi-value facet option for the ec_categories field, your search interface will allow users to filter results by selecting Electronics and/or Phones.
Example: Using the Replace action to trim part of a string
Let’s say you want to index product IDs in the product_id field.
If the API that you call returns ID values such as gid://Store/Product/12345, you need to trim the gid://Store/Product/ part to only index the actual product ID.
The following rule replaces gid://Store/Product/ with an empty string in values retrieved with the %[id] dynamic value.
So, if your API returns "id": "gid://Store/Product/12345", Coveo indexes 12345 in the product_id field.
"ExtendedMetadata": {
"product_id": {
"Expression": {
"value": "%[id]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Replace",
"OldValue": "gid://shopify/Product/",
"NewValue": ""
}
]
}
} |
In the product_id field, Coveo will index the result of the rule in this object. |
|
%[id] is a dynamic value that retrieves the metadata stored under id in your API response. |
|
Coveo will alter the metadata retrieved with %[id] according to the rules specified in the ExtendedMetadataActions array. |
DefaultValue (string)
Services > Endpoints > ExtendedMetadata > ExtendedMetadataActions > DefaultValue
When configuring your source, you may add rules instructing your source to alter the metadata it’s about to index.
In such case, if your rule object contains a Default action, it must also include the DefaultValue property.
DefaultValue is the default value to use if the metadata targeted by the rule is missing, that is, if the API returns a null value or an empty string ("").
Example: Replacing a missing metadata value with a default value
The following rule replaces a missing metadata value with a default value.
So, if your API returns "name_author": "" or "name_author": null, Coveo indexes "Unknown author" in the author field.
"ExtendedMetadata": {
"author": {
"Expression": {
"Value": "%[name_author]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "Unknown author",
}
]
}
} |
In the author field, Coveo will index the result of the rule in this object. |
|
%[name_author] is a dynamic value that retrieves the metadata stored under name_author in your API response. |
|
Coveo will alter the metadata retrieved with the %[name_author] according to the rules specified in the ExtendedMetadataActions array. |
Example: Filtering the metadata to return a single value
Let’s say you want to index product title translations in the product_title field.
However, if a translation is missing, you want to index the product title in the default language instead.
The following rule retrieves the title value from the localeTranslations object in your API response.
If the translation is missing, Coveo indexes the product title in the default language in the product_title field.
"ExtendedMetadata": {
"product_title": {
"Expression": {
"value": "%[localeTranslations[?(@.key == 'title')].value]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "%[title]"
}
]
}
} |
In the product_title field, Coveo will index the result of the rule in this object. |
|
This query retrieves the metadata stored under localeTranslations in your API response and filters it to only return the title value. |
|
Coveo will alter the metadata retrieved with the query according to the rules specified in the ExtendedMetadataActions array. |
Optionally, you can also add the FallbackValuesTrigger property to your ExtendedMetadataActions object.
FallbackValuesTrigger lets you define what DefaultValue should replace.
FallbackValuesTrigger (array)
Services > Endpoints > ExtendedMetadata > ExtendedMetadataActions > FallbackValuesTrigger
When configuring your source, you may add rules instructing your source to alter the metadata it’s about to index.
In such case, if your rule object contains a Default action, you can also include the FallbackValuesTrigger property.
FallbackValuesTrigger lets you define what DefaultValue should replace.
By default, if your API returns a null value or an empty string (""), Coveo replaces it with the string provided under DefaultValue.
However, if you know your API returns something different when the metadata is missing, like "None", you can use FallbackValuesTrigger to specify that "None" should be replaced with the DefaultValue.
Example
The following rule replaces a missing metadata value with a default value.
So, if your API returns "name_author": "None" or "name_author": "(Empty)", Coveo indexes "Unknown author" in the author field.
"ExtendedMetadata": {
"author": {
"Expression": {
"Value": "%[name_author]"
},
"ExtendedMetadataActions": [
{
"ActionName": "Default",
"DefaultValue": "Unknown author",
"FallbackValuesTrigger": [
"None", "(Empty)"
]
}
]
},
}, |
In the author field, Coveo will index the result of the rule in this object. |
|
%[name_author] is a dynamic value that retrieves the metadata stored under name_author in your API response. |
|
Coveo will alter the metadata retrieved with the %[name_author] according to the rules specified in the ExtendedMetadataActions array. |
StringSeparator (string)
Services > Endpoints > ExtendedMetadata > ExtendedMetadataActions > StringSeparator
The character or string that separates the hierarchical metadata values returned by your API.
Use this property when your API returns hierarchical values such as Men > Shoes > Sneakers and you use the FormatHierarchical action to reformat this information as Men;Men|Clothing;Men|Clothing|Shoes.
The new format allows you to leverage this metadata in a hierarchical facet.
Example
You use the FormatHierarchical action because you’re planning to add a hierarchical facet to your Coveo-powered search interface.
This action reformats your metadata in the way that’s required for the hierarchical facet to work properly.
Thanks to the StringSeparator property, Coveo can identify the hierarchy levels in the metadata.
So, if your API returns "category": "Men > Clothing > Shoes", Coveo indexes Men;Men|Clothing;Men|Clothing|Shoes in the ec_categories field.
Then, once you add a hierarchical facet based on the ec_categories field to your search interface, your search interface will allow users to filter results by selecting a category such as Men, and then a subcategory such as Shoes, and so on.
"ExtendedMetadata": {
"ec_categories": {
"Expression": {
"Value": "%[category]"
},
"ExtendedMetadataActions": [
{
"ActionName": "FormatHierarchical",
"StringSeparator": " > "
}
]
}
} |
In the ec_categories field, Coveo will index the result of the rule in this object. |
|
%[category] is a dynamic value that retrieves the metadata stored under category in your API response. |
|
Coveo will alter the metadata retrieved with %[category] according to the rules specified in the ExtendedMetadataActions array. |
|
Thanks to the StringSeparator property, Coveo can identify the hierarchy levels in the metadata. |
IndexingAction (object)
Whether to ignore or to retrieve an item when a condition is met.
When the Condition resolves to true, the Coveo crawler applies the specified action to the item.
Possible actions are Retrieve and Ignore, that is, the crawler can either index the item or ignore it.
An item ignored with IndexingAction isn’t indexed and therefore not visible in the Content Browser or a search interface.
However, its sub-items are sent to the index, unless otherwise specified in the ProcessingAction object.
If the condition resolves to false, the Coveo crawler falls back to the default action, which is Retrieve, and the specified item is indexed.
Basic example 1
With the following IndexingAction object, all items with an id greater than 5 are ignored.
Only items with an id of 5 or less are indexed.
"IndexingAction": {
"ActionOnItem": "Ignore",
"Condition": "%[id] > 5"
}Basic example 2
With the following IndexingAction object on an endpoint, all items that have a name metadata value other than myvalue are ignored.
The ProcessingAction object has the same condition.
As a result, a child item of the ignored items is also ignored if its name metadata value is not myvalue.
"IndexingAction": {
"ActionOnItem": "Ignore",
"Condition": "NOT %[raw.name]=='myvalue'"
},
"ProcessingAction": {
"ActionOnItem": "Ignore",
"Condition": "NOT %[raw.name]=='myvalue'"
} |
As stated under Condition, the metadata fields in your condition must either be defined in the Metadata object or referenced with raw. |
Realistic example
Let’s say you want to index store data, with full-time employee data as store subitems. However, in your API, the employee data isn’t directly linked to the store data. Instead, it’s linked to the store data through the department data.
Here’s an example of the data structure as it would be returned by your API:
Store 1
Department A
Employee 1
type: full time
Employee 2
type: full time
Department B
Employee 3
type: full time
Employee 4
type: part time
Employee 5
type: full time
Store 2
Department C
Employee 6
type: part time
Employee 7
type: full time
Employee 8
type: full time
Department D
Employee 9
type: full time
Department E
Employee 10
type: full time
Employee 11
type: full time
Employee 12
type: maternity leaveTo index employee items, you need to retrieve the parent department items first.
However, since you want don’t want to index the department data, you’ll use the IndexingAction object to have Coveo ignore it.
As a result, Coveo will index employee items as subitems of the store items, even if they don’t have a parent-child relationship in the API response.
Your source configuration is the following:
{
"Services": [
{
"Url": "https://api.com",
"Endpoints": [
{
"Path": "/search",
"Method": "GET",
"Headers": {
"x-api-key": "@ApiKey",
"Accept": "application/json",
"Host": "api.com"
},
"QueryParameters": {
"type": "grocery"
},
"ItemType": "Store",
"Uri": "%[url]",
"ClickableUri": "%[url]",
"Title": "%[name]",
"ModifiedDate": "%[updatedAt]",
"Body": "%[address]",
"Metadata": {
"id": "%[id]"
},
"SubItems": [
{
"ItemPath": "departments",
"ItemType": "Department",
"Uri": "%[url]",
"ClickableUri": "%[url]",
"Title": "%[name]",
"ModifiedDate": "%[updatedAt]",
"IndexingAction": {
"ActionOnItem": "Ignore"
},
"Body": "description",
"Metadata": {
"id": "%[id]"
},
"SubItems": [
{
"ItemPath": "employees",
"ItemType": "Employee",
"Uri": "%[url]",
"ClickableUri": "%[url]",
"Title": "%[name]",
"ModifiedDate": "%[updatedAt]",
"Body": "description",
"Metadata": {
"id": "%[id]",
},
"IndexingAction": {
"ActionOnItem": "Ignore",
"Condition": "NOT %[raw.type]=='full time'"
}
}
]
}
]
}
]
}
]
} |
This IndexingAction object is used to ignore Department items.
Since there’s no condition, all Department items will be ignored. |
|
This IndexingAction object is used to index only the Employee items that have full time as their type metadata value.
Employees items with a different type value will be ignored. |
|
As stated under Condition, the metadata fields in your condition must either be defined in the Metadata object or referenced with raw. |
The IndexingAction object supports the following properties:
-
ActionOnItem(required)
ActionOnItem (string, required)
Services > Endpoints > IndexingAction > ActionOnItem
Action to perform if the specified Condition resolves to true.
In the IndexingAction object, this applies to retrieved items.
Possible actions are Retrieve and Ignore, that is, the crawler can either index the item to make it searchable or ignore it.
In the ProcessingAction object, this applies to child items of an item.
Possible actions are Process and Ignore, that is, the crawler can either retrieve the subitems to make them searchable or ignore them.
Action Condition (string)
Services > Endpoints > IndexingAction > Condition
The Condition value is a condition that must resolve to true for the specified action to apply to the child items of an item.
The syntax to use is the same as for indexing pipeline extension conditions.
The metadata fields in your condition must either be defined in the Metadata object or referenced with raw.
You can specify an array if the metadata refers to a multi-value field in your repository.
Conditions can be assembled using the following operators: AND, OR, Exists, NOT, >, and <.
Parentheses are also supported to specify operation order.
Examples
-
"%[author_id]"is true if the item has anauthor_id. -
"NOT %[author_id]"is true if the item doesn’t have anauthor_id. -
"%[author_id] == 1234"is true if the itemauthor_idis1234. -
"%[author_ids] == [1,2,3,4]"is true if itemauthor_idsare1,2,3, and4. -
"%[author_id] OR %[author_name]"is true if the item has anauthor_idor anauthor_name. -
"%[author_id] AND %[author_name]"is true if the item has anauthor_idand anauthor_name. -
"%[author_id] > 123"is true if the itemauthor_idis greater than 123. -
"(%[author_id] OR %[author_sys_id]) AND %[author_name]"is true if the item has anauthor_idor anauthor_name, as well as anauthor_name.
ItemPath (string, required)
To index items or permissions, Coveo needs to know where to find this data in your API’s JSON response.
The ItemPath value should be d.results (see response structure).
Metadata (object)
A key-value list of the metadata to parse.
This object maps attributes from the integration object to the Coveo fields. Some ecommerce fields are created and mapped by default. You can add customs fields via the Fields (platform-ca | platform-eu | platform-au) page.
Each key represents the metadata name of the item, while its value is the value path (simple path or JSONPath) in the JSON response.
Typically, the value path consists of one or more dynamic values, since a static, hardcoded value would result in an identical piece of metadata for all items. However, you could choose to use a hardcoded value so that the corresponding Coveo field is filled even if the API doesn’t provide this information.
As an alternative to or in addition to the Metadata object, you can use the ExtendedMetadata object.
ExtendedMetadata lets you define rules to transform or customize pieces of metadata before indexing.
See ExtendedMetadata for details.
Example:
"metadata": {
"objecttype": "Product",
"ec_brand": "%[brand]",
"ec_name": "%[name]",
"name": "%[name]",
"ec_item_group_id": "%[code]",
"ec_shortdesc": "%[summary]",
"ec_description": "%[description]",
"ec_price": "%[europe1Prices.results[0].price]",
"ec_thumbnails": "https://accstorefront.commerce.ondemand.com%[thumbnail.URL]",
"ec_in_stock": "%[stock]",
"ec_images": "https://accstorefront.commerce.ondemand.com%[picture.URL]",
"ec_category": "%[supercategories.results[:].name]",
"ec_pricerange": "%[priceRange]",
"ec_configurable": "%[configurable]",
"ec_code": "%[code]",
"ec_url": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
}
Method (string enum)
The HTTP method to use to fetch the resource.
Possible values are GET and POST.
This property is not inheritable.
See the API’s documentation to determine which one you should use.
Example: "Method": "POST"
ModifiedDate (string)
The date on which the item was last modified. Although this property isn’t required in the JSON configuration, we recommend that you include it, as the Coveo-powered search pages use it for their default Sort by date option.
Typically, ModifiedDate has a dynamic value, since a static, hardcoded value would make dates identical for all items.
Example: "ModifiedDate": "%[updated]"
PayloadJsonContent (string)
The JSON content to send as a POST request body.
The Content-Type header is automatically set to application/json.
A PayloadJsonContent string cannot contain dynamic values if it’s located directly in an Endpoint or RefreshEndpoint object.
Since dynamic values are placeholders for data that Coveo extracts from your API response, an Endpoint or RefreshEndpoint query that contains a dynamic value is invalid.
Because it hasn’t made the query yet, Coveo doesn’t have an API response to use to resolve the dynamic value.
On the other hand, dynamic values are supported in a PayloadJsonContent string located within SubItems, SubQueries, and PermissionSubQueries arrays.
Since these arrays represent queries to make after the main Endpoint or RefreshEndpoint query, Coveo will be able to resolve the dynamic values using the API response from the main query and the item metadata (as well as the parent item’s metadata, if applicable).
|
|
Notes
|
You want to provide the following JSON content with your request:
{
"key1": "value1",
"key2": "@field=(\"value2\")",
"key3":
{
"subkey1": "subvalue1"
}
}Your source JSON configuration therefore contains the following:
"Endpoints": [
{
"Method": "POST",
"Path": "/rest/myendpoint/",
"PayloadJsonContent": "{\"key1\": \"value1\", \"key2\": \"@field=(\\\"value2\\\")\", \"key3\": {\"subkey1\": \"subvalue1\"}}"
}
]
PayloadParameters (object)
A key-value list of HTTP parameters to add to the payload.
Each value can be either a number, string, or Boolean.
This property can only be used in a POST request.
The parameters are sent as URL-encoded data in the request body.
The Content-Type header is automatically set to application/x-www-form-urlencoded.
A PayloadParameters object cannot contain dynamic values if it’s located directly in an Endpoint or RefreshEndpoint object.
Since dynamic values are placeholders for data that Coveo extracts from your API response, an Endpoint or RefreshEndpoint query that contains a dynamic value is invalid.
Because it hasn’t made the query yet, Coveo doesn’t have an API response to use to resolve the dynamic value.
On the other hand, dynamic values are supported in a PayloadParameters object located within SubItems, SubQueries, and PermissionSubQueries arrays.
Since these arrays represent queries to make after the main Endpoint or RefreshEndpoint query, Coveo will be able to resolve the dynamic values using the API response from the main query and the item metadata (as well as the parent item’s metadata, if applicable).
Dynamic time expressions are supported.
|
|
Notes
|
Examples:
Basic example
"PayloadParameters": {
"type": "post",
"expand": "true",
"id": 120
}
PermanentId (string)
The PermanentId is an alternative, permanent unique item identifier.
When you expect the item Uri to change, this ensures that Coveo Machine Learning (Coveo ML) recognizes the items despite their new URI and associates them to the old data.
So, in the future, if you modify or delete this source and reindex the same items, specifying the same item PermanentId format in your next configuration will ensure that ML doesn’t consider your items as new and rather picks up where it left off.
Dynamic values are supported.
Example:
{
"Uri": "%[coveo_url]/items/%[sys_id]",
"PermanentId": "%[sys_id]"
}
ProcessingAction (object)
This object works similarly to the IndexingAction object.
When the Condition resolves to true, the Coveo crawler applies the specified action on the corresponding child items of an item.
Possible actions are Process and Ignore, that is, the crawler can either index the subitems or ignore them.
A child item ignored with ProcessingAction isn’t indexed and therefore not visible in the Content Browser or a search interface.
If the condition resolves to false, it fallbacks to the default action, that is, Process.
Example 1
"ProcessingAction": {
"ActionOnItem": "Ignore",
"Condition": "%[id]==1"
}Example 2
With the following IndexingAction object on an endpoint, all items that have a name metadata value other than myvalue are ignored.
The ProcessingAction object has the same condition.
As a result, a child item of the ignored items is also ignored if its name metadata value is not myvalue.
"IndexingAction": {
"ActionOnItem": "Ignore",
"Condition": "NOT %[raw.name]=='myvalue'"
},
"ProcessingAction": {
"ActionOnItem": "Ignore",
"Condition": "NOT %[raw.name]=='myvalue'"
} |
As stated under Condition, the metadata fields in your condition must either be defined in the Metadata object or referenced with raw. |
Like the IndexingAction object, the ProcessingAction object supports the following properties:
-
ActionOnItem(required)
QueryParameters (object)
A key-value list of HTTP parameters to add to the query.
Each value can be either a number, string, or Boolean.
A QueryParameters object cannot contain dynamic values if it’s located directly in an Endpoint or RefreshEndpoint object.
Since dynamic values are placeholders for data that Coveo extracts from your API response, an Endpoint or RefreshEndpoint query that contains a dynamic value is invalid.
Because it hasn’t made the query yet, Coveo doesn’t have an API response to use to resolve the dynamic value.
On the other hand, dynamic values are supported in a QueryParameters object located within SubItems, SubQueries, and PermissionSubQueries arrays.
Since these arrays represent queries to make after the main Endpoint or RefreshEndpoint query, Coveo will be able to resolve the dynamic values using the API response from the main query and the item metadata (as well as the parent item’s metadata, if applicable).
|
|
Note
In sub-items and subqueries, should a dynamic value be invalid or unretrievable, the source skips the item, that is, the indexing process is dropped and Coveo moves on to the next item. In a permission subquery, however, an invalid or unretrievable dynamic value causes the source to stop crawling and to display an error in the Administration Console. |
Dynamic time expressions are supported.
Examples:
Basic example
"QueryParameters": {
"type": "post",
"expand": "true",
"id": 120
}Among other query parameters, the SAP source supports the following:
Filter (string)
Services > Endpoints > QueryParameters > Filter
An expression specifying the products to be indexed. It must be URL-encoded.
Example: "$filter": "catalogVersion/integrationKey eq 'Online%7CpowertoolsProductCatalog' and approvalStatus/code eq 'approved'"
Expand (string)
Services > Endpoints > QueryParameters > Expand
The list of entries associated with a single integration object or a collection of integration objects.
Example: "$expand": "supercategories,thumbnails,picture,thumbnail,unit,catalogVersion/catalog,europe1Prices,europe1Prices/currency,bundleTemplates,localizedAttributes,supercategories/localizedAttributes,approvalStatus"
RefreshEndpoints (array)
The RefreshEndpoints array is required for your source to perform refresh operations, which index only the items that have been added, updated, or deleted since the last update operation.
Refreshes are typically faster than rescan and rebuild operations, as they only process the items that have changed.
For more information on source update operations, see Refresh, rescan, and rebuild.
Coveo recommends you configure a RefreshEndpoints array for each endpoint in your source configuration.
Each object in the RefreshEndpoints array represents a query that Coveo will make to your API when conducting a source refresh operation.
By default, a refresh object inherits the properties defined in its parent endpoint object. So, in a refresh query, you only need to specify what should be different from the original indexing query.
Then, when you refresh your source, the properties defined in an object of the RefreshEndpoints array override the properties defined under Endpoints.
When you rescan or rebuild the source, however, the Endpoints properties apply.
Example
When refreshing your source, Coveo indexes only the published items that have been updated since the last refresh operation, as instructed by the following RefreshEndpoints array.
Since no other property is specified, Coveo will index these items exactly like during the initial indexing operation.
"Endpoints": [
{
"Path": "/api/now/table/kb_knowledge",
"Method": "GET",
"ItemPath": "result",
"ItemType": "kbknowledge",
"Uri": "%[coveo_url]/kb_knowledge/%[sys_id]",
"PermanentId": "%[sys_id]",
"ModifiedDate": "%[sys_updated_on]",
"ClickableUri": "%[coveo_url]/nav_to.do?uri=kb_knowledge.do?sys_id=%[sys_id]%26sysparam_view-ess",
"Title": "%[short_description]",
"Body": "%[text]",
"QueryParameters": {
"sysparm_query": "workflow_state=published"
},
"Metadata": {
"short_description": "%[short_description]",
"number": "%[number]",
"workflow_state": "%[workflow_state]",
},
"RefreshEndpoints": [
{
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters": {
"sysparm_query": "workflow_state=published^sys_updated_on>javascript:gs.dateGenerate(@RefreshDate)"
}
}
]
}
] |
The DateFormat property specifies the format of the date value in the query parameters. |
|
|
Note
A source refresh operation will index a change in a sub-item only if its parent item has also changed. |
Objects of the RefreshEndpoints array support the properties of Endpoints objects, plus the following:
Example:
"RefreshEndpoints": [
{
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters": {
"workflow_state": "published",
"since": "@RefreshDate"
}
},
{
"IsDeletionQuery": true,
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters": {
"lastModifiedDate": "@Now-7d",
"status": "Deleted"
}
},
{
"IsDeletionQuery":true,
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"Path": "/api/table/audit_delete",
"QueryParameters": {
"tablename": "kb_knowledge",
"since": "@RefreshDate"
}
},
{
"IsDeletedItem": "%[is_deleted]",
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters": {
"since": "@RefreshDate"
}
}
]
IsDeletionQuery (Boolean)
Services > Endpoints > RefreshEndpoints > IsDeletionQuery
Whether the defined query returns items to delete from the index.
Default value is false, which means that all retrieved items are added to the index or updated.
If the query returns items to add, update, and delete all at once, use the IsDeletedItem property instead.
IsDeletedItem (Boolean)
Services > Endpoints > RefreshEndpoints > IsDeletedItem
When a refresh query returns items to add, update, and delete all at once, this property allows you to specify which of these items should be deleted.
Default value is false, but it’s considered to be true if IsDeletionQuery is set to true.
You can provide a dynamic value (for example, %[has_been_deleted_flag]) to be evaluated for each item or a static value.
Example:
"%[is_deleted]" deletes all items that have a "is_deleted": true JSON property flagging them as deleted.
{
"IsDeletedItem": "%[is_deleted]",
"DateFormat": "\\'yyyy-MM-dd\\',\\'hh:mm:ss\\'",
"QueryParameters": {
"since": "@RefreshDate"
}
}
DeleteChildren (Boolean)
Services > Endpoints > RefreshEndpoints > DeleteChildren
Whether to delete children when the retrieved parent item is deleted.
Default value is false.
If the value is set to true, child items are deleted based on their Uri.
SubItems (array)
A list of sub-items to retrieve.
Each sub-item is represented by a configuration object under Endpoints.
An item can have many different sub-items, for example, a blog post containing comments and attachments as child items.
The SubItems object establishes a parent-child relationship between items.
There’s no restriction on the number of sub-items an item can have.
In addition, sub-items can also have sub-items, for example if blog post comments also have attachments.
In the application JSON response, these sub-items are nested within each other, and so should be your SubItems arrays in your source configuration.
To refer to the metadata of a parent item, prefix the metadata name with coveo_parent.
You may also need to use raw.
The sub-items configuration supports all properties from the Endpoints object configuration, except RefreshEndpoints.
In addition, it supports all other properties from the Services object configuration.
Examples:
When the sub-items to index are located under a different endpoint, you provide the Path to their location.
"SubItems": [
{
"Path": "/posts/%[coveo_parent.id]/comments",
"Method": "GET",
"ItemType": "Comment",
"Uri": "%[coveo_url]/repository/posts/%[coveo_parent.id]/comments/%[id]",
"ClickableUri": "%[coveo_url]/posts/%[coveo_parent.id]/comments/%[id]",
"Title": "%[title]",
"ModifiedDate": "%[updated]",
"Body": "%[content]",
"Metadata": {
"author": "%[author]",
"createdby" :"%[coveo_parent.author]",
"id": "%[id]"
}
}
]Alternatively, when the desired sub-items are located under the same endpoint as their parent items (for example, the child item data is nested within the parent item data), you omit the Path property and provide the sub-items' location using the ItemPath property.
"SubItems": [
{
"ItemPath": "comments",
"ItemType": "Comment",
"Uri": "%[coveo_url]/repository/posts/%[coveo_parent.id]/comments/%[id]",
"ClickableUri": "%[coveo_url]/posts/%[coveo_parent.id]/comments/%[id]",
"Title": "%[title]",
"ModifiedDate": "%[updated]",
"Body": "%[content]",
"Metadata": {
"author": "%[author]",
"createdby" :"%[coveo_parent.author]",
"id": "%[id]"
}
}
]
SubQueries (array)
Each object in the SubQueries array represents a subquery to execute on every item in order to fetch additional metadata and/or the item body.
Subqueries are especially useful when this information is located under a different path than that of the item.
To refer to the metadata of a parent item, prefix the metadata name with coveo_parent.
You may also need to use raw.
Example:
"SubQueries": [
{
"Path": "/posts/%[coveo_parent.id]/additionalinfo",
"Method": "GET",
"Body": "%[data.body]",
"Metadata": {
"created": "%[created]"
}
}
]The subquery configuration supports the following properties from the Endpoints object configuration:
-
ItemPath(required)
In addition, it supports the following properties from the Services object configuration:
It also contains the following properties:
ExecutionCondition (string)
Services > Endpoints > SubQueries > ExecutionCondition
In your subquery definition, you can provide an execution condition with a dynamic expression. This condition is evaluated before the subquery is made and determines whether Coveo should make the subquery. If the condition evaluates to false, the subquery is not made.
Execution conditions are useful to avoid making subqueries that would return no results or that would be irrelevant in a specific context. This reduces the load on your API and improves the crawling operation performance.
The syntax to use is the same as for indexing pipeline extension conditions.
The metadata fields in your condition must either be defined in the Metadata object or referenced with raw.
You can specify an array if the metadata refers to a multi-value field in your repository.
Conditions can be assembled using the following operators: AND, OR, Exists, NOT, >, and <.
Parentheses are also supported to specify operation order.
Examples
-
"%[author_id]"is true if the item has anauthor_id. -
"NOT %[author_id]"is true if the item doesn’t have anauthor_id. -
"%[author_id] == 1234"is true if the itemauthor_idis1234. -
"%[author_ids] == [1,2,3,4]"is true if itemauthor_idsare1,2,3, and4. -
"%[author_id] OR %[author_name]"is true if the item has anauthor_idor anauthor_name. -
"%[author_id] AND %[author_name]"is true if the item has anauthor_idand anauthor_name. -
"%[author_id] > 123"is true if the itemauthor_idis greater than 123. -
"(%[author_id] OR %[author_sys_id]) AND %[author_name]"is true if the item has anauthor_idor anauthor_name, as well as anauthor_name.
Example:
"SubQueries": [
{
"Path": "graphql.json",
"Method": "POST",
"PayloadJsonContent": "@ProductsTranslations",
"ExecutionCondition": "NOT(%[meta_field_ids] == '')",
"Metadata": {
"binding_mount": "%[$.data.translatableResourcesByIds.nodes[?(@.resourceId=='%[binding_mount_id]')].translations[0].value]"
}
}
] |
The subquery is only made if the meta_field_ids metadata is not empty. |
IsBinaryBody (Boolean)
Services > Endpoints > SubQueries > IsBinaryBody
Whether the subquery retrieves binary content as the body of the item.
Default value is false.
This property is available within SubQueries objects only.
It allows you to download a file (for example, XLS or PDF document) to use as the item body in a search result Quick view.
See Supported file formats for an exhaustive list of item types that Coveo can index.
Example:
"SubQueries": [
{
"Headers": {
"x-api-key": "@ApiKey"
},
"Path":"%[coveo_parent.url]",
"Method":"GET",
"IsBinaryBody":true,
"Body": "%[data.body]"
}
]|
|
Notes
|
IsThumbnail (Boolean)
Services > Endpoints > SubQueries > IsThumbnail
If the SubQueries object path leads to an image, set IsThumbnail to true to show this image as the thumbnail of an item.
The default value is false.
Supported thumbnail file formats are the following:
.bmp, .emf, .exif, .gif, .icon, .jpeg, .png, .tiff, .wmf
.
|
|
Notes
|
Title (string)
The title of the item. Although this property isn’t required in the JSON configuration, we recommend that you include it, as the Content Browser displays its value.
Typically, Title contains a dynamic value, since a static, hardcoded value would make all item titles identical.
Example: "Title": "%[title]"
Url (string, required)
Services > Url
The Url value is the service URL of your SAP Commerce Cloud OData2 GET API.
Example: "Url": "https://api.acmestore.com/odata2webservices"
Authentication (object)
Services > Authentication
If your API service requires Coveo to authenticate, you must enter the necessary credentials in the Authentication section of the Add an SAP source panel.
Then, in your JSON configuration, you must specify how Coveo should use these credentials to authenticate with the API.
If your API uses password-based or OAuth 2.0 authentication, you must use the Authentication object, which contains the information Coveo will use to authenticate to the service.
What if my API uses API keys?
If your API uses API keys for authentication, Coveo will typically need to provide such a key in an HTTP header, as a query parameter, or as a payload parameter.
You therefore don’t need an Authentication object in your JSON configuration.
In the Authentication object, you’ll use a placeholder starting with @ to refer to the credentials you entered in the panel.
This ensures that your credentials are encrypted rather than stored in clear text in the JSON configuration, where other Administration Console users could access them.
Then, when Coveo processes the JSON configuration, it replaces the placeholders with the actual credentials you entered in the panel.
The following table shows the objects and placeholders you must include in your configuration, depending on the type of authentication your API uses:
| Authentication type | Use placeholders in | Placeholders to use |
|---|---|---|
Basic, HTTP, Kerberos, or NTLM (password-based) |
|
|
OAuth 2.0 |
|
|
API key |
|
|
As indicated in the table above, API key placeholders don’t appear in the Authentication object.
If your API authenticates with API keys, see the sections listed in the table for examples with API key placeholders.
Authentication properties specified at the service level apply to the child endpoints, and sub-queries underneath them.
Typically, you don’t need to override the configuration you entered at the service level further down in the JSON configuration, as the same authentication method applies to the entire application content.
Examples:
Basic, HTTP, Kerberos, or NTLM
If your API uses password-based authentication, you must enter the username and password in the Authentication section of the Add an SAP source panel.
Then, in your JSON configuration, you must use the @Username and @Password placeholders to refer to these credentials.
Your Authentication object therefore looks like this:
"Authentication": {
"Username": "@Username",
"Password": "@Password",
"ForceBasicAuthentication": "true"
}OAuth 2.0
If your API uses OAuth 2.0 authentication, you must enter the client ID, client secret, and refresh token in the Authentication section of the Add an SAP source panel.
Then, in your JSON configuration, you must use the @ClientId, @ClientSecret, and @RefreshToken placeholders to refer to this information.
Your Authentication object could therefore look like this:
"Authentication": {
"OAuth": {
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
}
}
}
}
}The Authentication object supports the following properties:
-
OAuth(required if authenticating with OAuth 2.0)
Username (string)
Services > Authentication > Username
Username used to log in to the service.
You can use @Username to retrieve the value specified in the Add an SAP Source panel.
Password (string)
Services > Authentication > Password
Password used to log in to the service.
This property can be empty even though a Username is specified.
You can use @Password to retrieve the value specified in the Add an SAP Source panel.
Domain (string)
Services > Authentication > Domain
Domain name to use when authenticating to a NTLM or Kerberos protected server.
ForceBasicAuthentication (Boolean)
Services > Authentication > ForceBasicAuthentication
Whether to force a basic HTTP header in the request.
The default value is false.
OAuth (object)
Services > Authentication > OAuth
If your application requires OAuth 2.0 authentication, add an OAuth object to your source configuration.
Your OAuth object must contain a Query object.
Coveo will use the information in the Query object to make an authentication query to your application.
Optionally, your OAuth object may also contain a Response object.
In this object, you may include properties such as AccessToken, TokenType, and ExpiresIn, whose values indicate where Coveo will find the corresponding information when parsing the API’s response.
The Response object may also include the AuthorizationHeader property, whose value is the name of the authorization header that Coveo should use when querying your API for content.
|
|
Enter your client ID, client secret, and refresh token in the Add an SAP source panel and use the This allows you to keep your confidential information encrypted and obfuscated, thus preventing other Administration Console users to access it. 
|
Example:
"OAuth": {
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
},
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
}The OAuth object supports the following properties:
-
Query(required)
Query (object, required)
Services > Authentication > OAuth > Query
If your application requires OAuth 2.0 authentication, you must add an OAuth object to your source configuration.
Your OAuth object must contain a Query object.
Coveo will use the information in the Query object to make an authentication query to your application.
|
|
Enter your client ID, client secret, and refresh token in the Add an SAP source panel and use the This allows you to keep your confidential information encrypted and obfuscated, thus preventing other Administration Console users to access it.
|
Example:
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}The Query object supports the following properties:
-
RefreshUrl(required)
RefreshUrl (string, required)
Services > Authentication > OAuth > Query > RefreshUrl
If your application requires OAuth 2.0 authentication, your Query object must include a RefreshUrl property.
The RefreshUrl value must be a URL that Coveo will use to refresh its access token.
Example
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}OAuth query Headers (object)
Services > Authentication > OAuth > Query > Headers
If your application requires OAuth 2.0 authentication, your Query object can optionally include a Headers object.
When making an authentication query to your application, Coveo will add these HTTP headers to its request.
|
|
Note
Coveo automatically sets the Content-Type header.
Therefore, you don’t need to add it to the |
|
|
To keep your header value encrypted and obfuscated, use a placeholder in your 
|
OAuth query Method (string enum)
Services > Authentication > OAuth > Query > Method
If your application requires OAuth 2.0 authentication, your Query object can optionally include a Method property.
The Method property specifies the HTTP method that Coveo should use when refreshing its token.
Default is POST.
Example
"Query": {
"RefreshUrl": "http://example.com/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}OAuth query Parameters (object)
Services > Authentication > OAuth > Query > Parameters
If your application requires OAuth 2.0 authentication, your Query object can optionally include query parameters for Coveo to use.
Each object in the Parameters object represents a parameter to include in Coveo’s authentication request to your application.
Each object should contain a "Type" and a "Value" property.
The Type property indicates how to include the parameter in the request.
Possible values for this property are query or payload.
If you include a refresh_token object, you must add a IsRefreshToken property with a value of true to this object.
Moreover, if this refresh token is single-use, make sure to specify it by adding "UsingSingleUseRefreshToken": true to your OAuth object.
|
|
Enter your client ID, client secret, and refresh token in the Add an SAP source panel and use the This allows you to keep your confidential information encrypted and obfuscated, thus preventing other Administration Console users to access it.
|
Example:
"Authentication": {
"OAuth": {
"UsingSingleUseRefreshToken": true,
"Query": {
"RefreshUrl": "https://auth.example.com/oauth/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
}
}
}
}
}
Response (object)
Services > Authentication > OAuth > Response
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
The Response object contains, among other things, the properties that Coveo must extract from the authentication server’s response.
The value of each property represents the path to the corresponding value in the response.
In the Response object, you may also enter other authentication parameters, such as the AuthorizationHeader and SupportsRefreshToken properties.
Example:
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}Then your Response object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}The Response object supports the following properties:
AccessToken (string)
Services > Authentication > OAuth > Response > AccessToken
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
Coveo will then extract the information from the API’s response using the values specified in the Response object.
In the Response object, you can add the AccessToken property to specify the path to the access token in the API’s response.
Default is access_token.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}Then your Response object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
SupportsRefreshToken (Boolean)
Services > Authentication > OAuth > Response > SupportsRefreshToken
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
In the Response object, you can add the SupportsRefreshToken property to indicate whether your application’s OAuth 2.0 flow supports refresh tokens.
Default is true.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}Then your Response object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
RefreshToken (string)
Services > Authentication > OAuth > Response > RefreshToken
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
Coveo will then extract the information from the API’s response using the values specified in the Response object.
In the Response object, you can add the AccessToken property to specify the path to the refresh token in the API’s response.
Default is refresh_token.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}Then your Response object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
ExpiresIn (string)
Services > Authentication > OAuth > Response > ExpiresIn
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
Coveo will then extract the information from the API’s response using the values specified in the Response object.
In the Response object, you can add the ExpiresIn property to specify the path to the refresh token expiration delay in the API’s response.
Default is expires_in.
If the API’s response doesn’t contain an expiration delay, use the ExpiresInDefaultValue property instead.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}Then your Response object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
ExpiresInDefaultValue (number)
Services > Authentication > OAuth > Response > ExpiresInDefaultValue
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
Coveo will then extract the information from the API’s response using the values specified in the Response object.
Typically, the authentication server will return an expiration delay for its refresh token, which you can extract with the ExpiresIn property.
However, if the API’s response doesn’t include an expiration delay, you can add the ExpiresInDefaultValue property to your Response object.
The ExpiresInDefaultValue value specifies the number of seconds after which Coveo should consider the refresh token expired.
Default is 3600.
Example:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresInDefaultValue": 1800,
"TokenType": "token_type"
}
TokenType (string)
Services > Authentication > OAuth > Response > TokenType
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
Coveo will then extract the information from the API’s response using the values specified in the Response object.
In the Response object, you can add the ExpiresIn property to specify the path to the token type in the API’s response.
Default is token_type.
If the response doesn’t specify a token type or if it’s invalid or inappropriate, Coveo will use Bearer by default.
If you want a different default value, use the TokenTypeDefaultValue property instead.
Example
If the authentication server’s response looks like this:
HTTP/1.1 200 OK
Content-Type: application/json{
"access_token": "jOs0PoLKm61sd6h49",
"refresh_token": "QpMas-tuI9bvFp01-Xxpl04gC_0m",
"token_type": "bearer",
"expires_in": 86400
}Then your Response object should look like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type"
}
TokenTypeDefaultValue (string)
Services > Authentication > OAuth > Response > TokenTypeDefaultValue
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
Coveo will then extract the information from the API’s response using the values specified in the Response object.
Typically, the authentication server will return a token type, which you can extract with the TokenType property.
If the API’s response doesn’t specify a token type, or if the token type is invalid or inappropriate, Coveo will use Bearer by default.
However, if you want Coveo to use a different default token type, you can specify it by the TokenTypeDefaultValue property to your Response object.
AuthorizationHeader (string)
Services > Authentication > OAuth > Response > AuthorizationHeader
If your application requires OAuth 2.0 authentication, you can optionally add a Response object to your OAuth object.
In the Response object, you can add the AuthorizationHeader property to indicate the name of the authorization header that Coveo should use to provide its access token when querying your API for content.
Refer to your API’s documentation for the header name to use.
Default is Authorization.
|
|
Notes
Coveo automatically adds the specified authorization header to its content requests to your application.
Do not add this header to the |
Example:
If your API documentation specifies that the access token must be provided in a header named X-ACCESS-TOKEN, then your Response object should specify it like this:
"Response": {
"AccessToken": "access_token",
"SupportsRefreshToken" : "true",
"RefreshToken": "refresh_token",
"ExpiresIn": "expires_in",
"TokenType": "token_type",
"AuthorizationHeader": "X-ACCESS-TOKEN"
}As a result, when making content requests to your application, Coveo will automatically include an X-ACCESS-TOKEN header with the access token value.
{
"headers": {
"Content-Type": "application/json",
"X-ACCESS-TOKEN": "123456789abcdef"
}
}
UsingSingleUseRefreshToken (Boolean)
Services > Authentication > OAuth > UsingSingleUseRefreshToken
Set UsingSingleUseRefreshToken to true to indicate that the refresh token used by the source to authenticate to your application is single-use.
Default is false.
A single-use refresh token expires after it’s been used.
When setting UsingSingleUseRefreshToken to true, make sure to include a refresh_token query parameter in your OAuth query.
Example
"Authentication": {
"OAuth": {
"UsingSingleUseRefreshToken": true,
"Query": {
"RefreshUrl": "https://auth.example.com/oauth/token",
"Method": "POST",
"Parameters": {
"grant_type": {
"Type": "Payload",
"Value": "refresh_token"
},
"client_id": {
"Type": "Payload",
"Value": "@ClientId"
},
"client_secret": {
"Type": "Payload",
"Value": "@ClientSecret"
},
"refresh_token": {
"Type": "Payload",
"Value": "@RefreshToken",
"IsRefreshToken": true
}
}
}
}
}
Headers (object)
Services > Headers
A key-value list of HTTP headers to add to the query. Each value can be either a number, string, or Boolean.
Your Headers object must include a unique Application Interface Key (Application Interface Register, AIR).
Refer to the SAP documentation for details.
The Headers object is inheritable.
Examples:
Basic example
"Headers": {
"Custom-Header-Name": "myValue",
"Application-Interface-Key": "xxxxxxxx"
}
Paging (object)
Services > Paging
In the Paging object, specify how you want the content to be paged.
When building your object from scratch, we recommend starting with the OffsetType property.
The Paging object is inheritable.
That is, when you set it at the service level, the Paging object applies to all endpoints in this service by default.
However, you can disable inheritance using the DoNotInherit property at the service level.
This may help speed up the crawling process.
You can also override the service-level configuration by setting the Paging object again at the endpoint level.
Sub-items inherit this property.
Sub-queries don’t inherit this property.
Example:
"Paging": {
"PageSize": 20,
"OffsetStart": 0,
"OffsetType": "page",
"Parameters": {
"Limit": "limit",
"Offset": "page"
},
"DoNotInherit": true
}The Paging object supports the following properties:
-
OffsetType(required) -
PageSize(required)
OffsetType (string enum, required)
When configuring your source to request paginated content, you must specify how your API paginates its content. Coveo needs this information to build its page request URLs in the appropriate format. You can typically find it in your API’s documentation.
The OffsetType property is a string enum that specifies the type of pagination your API uses.
The value to specify depends on your API’s pagination technique, just like the properties you’ll need to add to your Paging object.
| Pagination technique |
OffsetType value |
Properties to add to the Paging object |
|---|---|---|
Page-based |
|
|
Offset-based |
|
|
URL-based |
|
|
Cursor-based |
|
Examples:
Page-based pagination
Your API expects request URLs in a format such as http://example.com/api/item?limit=100&pageNumber=0 for the first page, and http://example.com/api/item?limit=100&pageNumber=1 for the second page.
With these URLs, Coveo would request the first page (page 0), and then increment the page number by 1 to request the second page, and so on.
In this case, your Paging object should look like this:
{
"Paging": {
"PageSize": 100,
"OffsetStart": 0,
"OffsetType": "page",
"Parameters": {
"Limit": "limit",
"Offset": "pageNumber"
}
}
}Offset-based pagination
Your API expects request URLs in a format such as http://example.com/api/item?limit=100&start=0 for the first page, and http://example.com/api/item?limit=100&start=100 for the second page.
With these URLs, Coveo would request the 100 first items starting from item 0, and then the 100 next items, starting from item 100.
In this case, your Paging object should look like this:
{
"Paging": {
"PageSize": 100,
"OffsetStart": 0,
"OffsetType": "item",
"Parameters": {
"Limit": "limit",
"Offset": "start"
}
}
}URL-based pagination
Your API expects request URLs in a format such as http://example.com/api/item?limit=8 for the first page, and http://example.com/api/item?page=3d170d80d8n3n2342c328s for the second page.
In this context, Coveo would request the first page, and then the next page using the URL provided in the nextPage value in the JSON response.
In this case, if your API returns the following first page:
{
"nextPage": "http://example.com/api/item?page=3d170d80d8n3n2342c328s",
"items": [
{ "id": 1, "name": "Caroline" },
{ "id": 2, "name": "Marcella" },
{ "id": 3, "name": "Susie" },
{ "id": 4, "name": "Rhonda" },
{ "id": 5, "name": "Wendy" },
{ "id": 6, "name": "Barbara" },
{ "id": 7, "name": "Deirdre" },
{ "id": 8, "name": "Lynda" }
]
}Then your Paging object should look like this:
{
"Paging": {
"PageSize": 8,
"NextPageKey": "nextPage",
"OffsetType": "url",
"Parameters": {
"Limit": "limit"
}
}
}Cursor-based pagination
Your API expects request URLs in a format such as http://example.com/api/item?limit=8 for the first page, and http://example.com/api/item?limit=8&nextPageToken=3d170d80-7b3b-4377 for the second page.
With these URLs, Coveo would request the 8 first items, and then the next page using the token provided in the NextPageId value in the JSON response.
In this case, if your API returns the following first page:
{
"nextPageId": "3d170d80-7b3b-4377",
"items": [
{ "id": 1, "name": "Caroline" },
{ "id": 2, "name": "Marcella" },
{ "id": 3, "name": "Susie" },
{ "id": 4, "name": "Rhonda" },
{ "id": 5, "name": "Wendy" },
{ "id": 6, "name": "Barbara" },
{ "id": 7, "name": "Deirdre" },
{ "id": 8, "name": "Lynda" }
]
}Then your Paging object should look like this:
{
"Paging": {
"PageSize": 8,
"NextPageKey": "NextPageId",
"OffsetType": "cursor",
"Parameters": {
"Limit": "limit",
"Offset": "nextPageToken"
}
}
}
PageSize (number, required)
The number of items to fetch per page.
Example: With paging URL https://example.com/api/item?position=0&quantity=50, your JSON configuration should include "PageSize": 50.
DoNotInherit (Boolean)
Whether to prevent your Paging configuration from being inherited.
This can be useful for preventing paging on PermissionSubqueries, which is known to slow down the crawling process.
The default value is false.
NextPageKey (string)
When the OffsetType value is "url" or "cursor", provide the path (simple path or JSONPath) to the key representing the value of the next page reference, that is, either the next page URL or the next page cursor.
The NextPageKey property isn’t required when the OffsetType value is page or item.
Example: "NextPageKey": "NextPageLink"
OffsetStart (number)
Offset of the first page to fetch.
This property is required when the OffsetType value is "page" or "item".
Example: Your first paging URL is https://example.com/api/item?position=0&quantity=50.
If you want to retrieve all items except items 1 to 50, your paging configuration must include "OffsetStart": 1.
Paging Parameters (object)
In the paging Parameters object, you must provide the names of the query string fields in your web application pagination URL that are required to support pagination.
The properties expected in this object are Limit and Offset.
However, Offset isn’t required when the OffsetType value is "url" since the complete URL is already retrieved in the JSON response.
Example:
"Parameters": {
"Limit": "limit",
"Offset": "page"
}The Parameters object supports the following properties:
-
Limit(required)
Limit (string, required)
Services > Paging > Parameters > Limit
In the query string, the field whose value is retrieved with the PageSize parameter.
Example: With paging URL https://example.com/api/item?position=0&quantity=50, your JSON configuration should include "Limit": "quantity".
FetchNextPageUntilNoResult (Boolean)
Services > Paging > Parameters > FetchNextPageUntilNoResult
The FetchNextPageUntilNoResult property indicates whether Coveo will try to fetch pages until no result is returned, or until the application API returns an error.
Default value is true.
The FetchNextPageUntilNoResult property is an alternative to the TotalCountHeaderKey and TotalCountKey properties.
|
|
Unless you set this property to |
Example:
"Paging": {
"OffsetType": "page",
"PageSize": 25,
"Parameters": {
"FetchNextPageUntilNoResult": true
}
}
Offset (string)
Services > Paging > Parameters > Offset
In the query string, the field whose value is retrieved with the OffsetStart property.
Example: Your first paging URL is https://example.com/api/item?position=0&quantity=50.
If you want to ignore items 1 to 50 and retrieve items 51 and over, your paging configuration must include "Offset": "position" and "OffsetStart": 1.
TotalCountHeaderKey (string)
The name of an HTTP header whose value represents the total number of items retrieved through the API call. This property is useful when Coveo makes an API call to a page that doesn’t exist, causing an error to be returned.
Example: "TotalCountHeaderKey": "x-total-count"
TotalCountKey (string)
The path (simple path or JSONPath) to a response body property representing the total number of items retrieved through the API call. This property is useful when Coveo makes an API call to a page that doesn’t exist, causing an error to be returned.
Example: "TotalCountKey": "retrievedContent.totalCount"
Permissions (array)
Services > Permissions
Each object in the Permissions array represents a permission level.
In turn, permission levels contain one or more PermissionsSets arrays.
The Permissions array is required when you select the Same users and groups as in your content system source content security option.
It contains the permission levels and permission sets to apply to the items indexed through this service.
The members listed in these levels and sets are either allowed or denied access to the indexed items.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
This property is inheritable.
Example:
"Permissions": [
{
"Name": "Permission Level 1",
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "jsmith@example.com",
"Type": "User",
"AdditionalInfo": {
"title": "%[customMetadata]"
}
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": false
}
]
}
]The Permissions array supports the following properties:
PermissionsSets (array)
Services > Permissions > PermissionsSets
The PermissionsSets array specifies who can or can’t access the indexed items.
Each object in the array describes a single permission set, which can contain lists of allowed and denied members.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
Objects in the PermissionsSets array support the following properties:
Permission set Name (string)
Services > Permissions > PermissionsSets > Name
The Name property indicates the name of the permission set.
AllowedMembers and DeniedMembers arrays
Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers
Each object in the AllowedMembers and DeniedMembers arrays represents a member security identity that should be allowed or denied to access the indexed content.
|
|
Note
If a user is listed under both |
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
You index text items written by different people.
The metadata of each item contains the key author, and, in this system, authors are identified by their email address.
These addresses are therefore expected as values of the Name field.
With the following configuration, each user in your organization can access the items they have created through your Coveo-powered search interfaces.
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "%[author]",
"Type": "User",
"PermissionType": "config1",
"Condition": "%[field] == value",
"AdditionalInfo": {
"title": "%[customMetadata]"
}
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": false
}
]AllowedMembers and DeniedMembers arrays support the following properties:
-
Name(required)
Member Name (string, required)
Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > Name
The name of the member security identity.
Email addresses and Active Directory usernames are supported.
Dynamic values are supported.
Member Type (string enum, required)
Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > Type
The type of member.
Allowed values are User, Group, and VirtualGroup.
For more information on these types of security identities, see Coveo management of security identities and item permissions.
Permission Condition (string)
Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > Condition
The Condition value is a condition that must resolve to true for the security identity specified in Name to be resolved, that is, extracted by the security identity provider.
As a result, the permission applies.
When the condition resolves to false, the security identity is not resolved and the corresponding permission doesn’t apply.
Adding a condition to your AllowedMembers or DeniedMembers object is optional.
To write your condition, use the same syntax as for indexing pipeline extension conditions.
Conditions can be assembled using the following operators: AND, OR, Exists, NOT, >, and <.
Parentheses are also supported to specify operation order.
The metadata fields in your condition must either be defined in the Metadata object of the source configuration or referenced with raw.
You can specify an array if the metadata refers to a multi-value field in your repository.
Dynamic values are also supported.
You index a list of available and discontinued products, and your public search interface is used by both your employees and your customers. On this search interface, your employees can log in to see additional content, while customers can’t log in.
You want to hide your discontinued products from your customers only, so you use the Condition property to apply a permission to these items.
As a result, all products with Discontinued: yes in their metadata are visible to members of the *@mycompany.com group only, that is, your employees, while non-discontinued products are available to all search interface users.
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "*@mycompany.com",
"Type": "Group",
"PermissionType": "config1",
"Condition": "%[discontinued] == yes"
}
],
"DeniedMembers": null,
"IsAnonymousAllowed": false
}
]For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
PermissionType (string)
Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > PermissionType
This property is relevant only when you provide a permission configuration.
The PermissionType is a key representing the configuration on how to extract all relationships of the security identity specified in Name.
The security identity provider uses this configuration when processing the security identity.
In the Content Security tab of the Add/Edit an SAP Source panel, you’ll need to select the Same users and groups as in your content system, and then to provide a JSON permission configuration detailing how to retrieve the relationships of each security identity and how to index this data.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
AdditionalInfo (object)
Services > Permissions > PermissionsSets > AllowedMembers/DeniedMembers > AdditionalInfo
The AdditionalInfo object is relevant only when you provide a permission configuration.
AdditionalInfo can be used to enrich a security identity with custom information to make it unique.
For example, if two users have the same name, AdditionalInfo lets you specify more information about each user, like their employee ID.
This custom information is part of the security identity, just like its name and its type.
Each key in the AdditionalInfo object represents the name of a piece of metadata, while the corresponding value is the value path (simple path or JSONPath) in your API’s JSON response.
Dynamic values are supported.
When writing your permission configuration, you can use coveo_parent to refer to the information retrieved with the AdditionalInfo object of a permission subquery in your source configuration.
Example
Your source configuration contains the following Permissions object, which indicates that members of the Support Agents group should be allowed to access the indexed content:
"Permissions": [
{
"Name": "Permission Level 1",
"PermissionsSets": [
{
"Name": "Permission Set 1",
"AllowedMembers": [
{
"Name": "Support Agents",
"Type": "Group",
"PermissionType": "MyPermissionType2",
"AdditionalInfo": {
"group_id": "00841"
}
}
],
}
]
}
]To extract the members of this group, you provide the following permission configuration.
Therefore, the security identity provider will use the group_id you specified in the source configuration to make an API request and obtain the group members.
Then, the security provider will refer to the user instructions to retrieve the desired information on the extracted group members.
"identities": {
"group": {
"permissionSubQueries": [
{
"path": "/group/%[coveo_parent.group_id]/members",
"method": "GET",
"itemPath": "members",
"permissionType": "user",
"name": "%[username]",
"type": "User",
"additionalInfo": {
"user_id": "%[user_id]"
}
}
]
},
"user": {
"permissionSubQueries": [
{
"path": "/users/%[user_id]",
"method": "GET",
"itemPath": "user",
"name": "%[email]",
"type": "User"
}
]
}
}
IsAnonymousAllowed (Boolean)
Services > Permissions > PermissionsSets > IsAnonymousAllowed
The IsAnonymousAllowed property indicates whether anonymous, that is, unauthenticated users are allowed to access the retrieved items.
Default value is false.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
PermissionsFromMetadata array
Services > Permissions > PermissionsSets > PermissionsFromMetadata
This array is relevant only when you provide a permission configuration.
The PermissionsFromMetadata array is an alternative to the PermissionSubQueries array, as it fetches permission data, typically the security identities that are allowed or denied access to the item.
This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo.
Include a PermissionsFromMetadata array in your source configuration when the permission data appears in the metadata of an item.
Conversely, a PermissionSubQueries array should be used when an item and its permission data must be retrieved with two different queries.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
Objects of the PermissionsFromMetadata array support:
PermissionSubQueries array
Services > Permissions > PermissionsSets > PermissionSubQueries
This array is relevant only when you provide a permission configuration.
Each object in the PermissionSubQueries array represents a subquery to execute on every item to fetch permission data, typically the security identities that are allowed or denied access to the item.
This data is crucial to retrieve if you want to replicate the application’s permission system in Coveo.
Each object in the PermissionSubQueries array inherits the paging properties specified at the parent endpoint or service level.
The PermissionSubQueries array is an alternative to the PermissionsFromMetadata array.
Include a PermissionSubQueries array in your source configuration when an item and its permission data must be retrieved with two different queries.
Conversely, the PermissionsFromMetadata array should be used when the permission data appears in the metadata of an item.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
Objects of the PermissionSubQueries array support:
-
The following properties from the
Endpointsobject configuration:-
ItemPath(required)
-
-
The following properties from objects of the
AllowedMembersandDeniedMembersarrays: -
The following property:
Example:
"permissionSubQueries": [
{
"ItemPath": "permissions",
"Path": "%[productid]/permissions",
"Method": "GET",
"QueryParameters": {},
"Name": "%[name]",
"Type": "group",
"PermissionType": "config1",
"Condition": "%[field] == value",
"IsAllowedMember": true,
"AdditionalInfo":
{
"PermissionID": "%[id]",
"PermissionName": "%[name]"
}
}
]
IsAllowedMember (Boolean)
Services > Permissions > PermissionsSets > PermissionSubQueries > IsAllowedMember
The IsAllowedMember property indicates whether the specified member is allowed to access the item.
Default value is false.
For more information on sources that index permissions and on how Coveo handles these permissions, see Coveo management of security identities and item permissions.
RetryableHttpErrorCodes (string)
Services > RetryableHttpErrorCodes
When crawling your content, Coveo may encounter an HTTP error. By default, this stops the indexing process, and an error appears on the Sources (platform-ca | platform-eu | platform-au) page.
However, some APIs may be unreliable and expected to occasionally return a temporary error.
In such case, you can use the RetryableHttpErrorCodes property to have Coveo retry the request and continue crawling.
This may be especially useful when the temporary error code returned by your API is non-standard.
Use a semicolon (;) to separate error codes.
Example: "RetryableHttpErrorCodes": "404;418".
This property is inheritable.
When you set it at the service level, it will apply to all endpoints in this service, and subqueries and sub-items in these endpoints.
You can override the service-level configuration by setting the RetryableHttpErrorCodes property again at the endpoint or subquery level.
RetryableHttpErrorCodes is an alternative to the SkippableErrorCodes property.
SkippableErrorCodes (string)
Services > SkippableErrorCodes
When crawling your content, Coveo may encounter an HTTP error. By default, this stops the indexing process, and an error appears on the Sources (platform-ca | platform-eu | platform-au) page.
However, some APIs may be expected to always return an error in a specific circumstance and to never return items after this error.
In such case, you can use the SkippableErrorCodes property to have Coveo ignore the error and continue crawling.
For example, let’s say you want to index subitems that may or may not exist depending on the parent item. If you know your API returns a 404 error when a subitem doesn’t exist, add the following to your configuration. Coveo will ignore the error and move on with the indexing process.
"SkippableErrorCodes": "404"Similarly, you can use this property if your API returns a 400 error when pagination can’t return any more items.
Use a semicolon (;) to separate error codes.
For example: "500;404;403".
|
|
We strongly discourage using the For example, if your API sometimes returns items and sometimes returns an error instead, you should not use this property to avoid interruptions of the indexing process. If a temporary error is ignored, items that were returned and indexed before the error will be considered as deleted. As a result, Coveo will delete these items from your index, and they won’t appear in your search interface anymore. The missing items will only be reindexed at the next update operation where Coveo doesn’t encounter the error. If you know your API is unreliable, you should instead use the |
This property is inheritable.
When you set it at the service level, it will apply to all endpoints in this service, and subqueries and sub-items in these endpoints.
You can override the service-level configuration by setting the SkippableErrorCodes property again at the endpoint or subquery level.
Very useful
Not really
Very useful
Not really