RSS source JSON modification

This is for:

Developer System Administrator

Many source configuration parameters can be set through the user interface. Others, such as rarely used or new parameters, must be configured in the Edit configuration with JSON panel. To access this panel from the Sources (platform-ca | platform-eu | platform-au) page, click the source, and then click Edit configuration with JSON in the More menu.

This article explains how to configure RSS source parameters, whether they’re already listed in the JSON or not.

Configuring listed and unlisted parameters

Changing a parameter value in the source JSON configuration | Coveo

If the parameter you want to change is already listed in the parameters section of the source JSON configuration, just modify its value in the JSON configuration.

If the parameter isn’t listed in the parameters section, copy the entire parameter example object from the Reference section below and paste it into that section. Then, update the value in the JSON configuration, if necessary.

Important

If a parameter has a value attribute that contains sensitive information, set the sensitive attribute to true. Otherwise, the value will appear in clear text in the JSON configuration.
Tip

Document the changes you make to the source JSON configuration in the Comments area below the JSON configuration. This ensures that you can easily revert to a previous configuration if needed.

Reference

This section provides information on the RSS source parameters that you can only modify through the JSON configuration.

If a JSON configuration parameter isn’t documented in this article, configure it through the source edition panel instead.

AllowedDeletionPercentage (Integer)

This parameter specifies the maximum allowed percentage of source items that can be deleted from the index at the end of a rescan. If the actual percentage of source items to delete exceeds this value, no items are deleted from the index. By default, this parameter is set to 100, which means that all source items can be deleted.

The purpose of this parameter is to prevent accidental mass item deletions. This can occur, for example, because of an improper source configuration or if the content to index was moved.

For more information about this parameter and its usage, see Forbid item deletion based on a percentage condition.

Example

You can set AllowedDeletionPercentage to 10 in the JSON configuration of your source, as shown in the snippet below. With this configuration, if Coveo detects that more than 10% of the items are flagged for deletion during a rescan, deletion will be blocked. The status on the Sources (platform-ca | platform-eu | platform-au) page will show your source in error, and the error details will indicate the actual percentage of items that were flagged for deletion versus the allowed percentage (in this case, 10%).

"AllowedDeletionPercentage": {
  "sensitive": false,
  "value": "10"
}

DateFormat (String | Null)

Specifies the format to use when parsing date element values in the feed. The format must not include time-zone and offset information.

Important

The default value is null, as the RSS source can automatically recognize and parse most date formats associated with the supported RSS feed formats. Specify a DateFormat value and retry indexing only if the date field value of your indexed items is incorrect.
Example
"DateFormat": {
  "sensitive": false,
  "value": "YYYY-MM-DDThh:mm:ss"
}

IndexRssFeedUrl (Boolean)

Determines whether the RSS feed URL is indexed. The default value is false.

If you set this parameter to true and rebuild, your source content will include an item whose Clickable URI is the Feed URL. The File type value of this item will be rssfeed.

Example
"IndexRssFeedUrl": {
  "sensitive": false,
  "value": "true"
}

MetadataToExcludeFromEtag (Array)

The list of metadata names to exclude from Coveo’s ETag calculation.

By default, Coveo generates an item ETag value using the entire content of the <item> or <entry> element in the feed. The crawler compares the current item’s ETag value with the one obtained during the most recent source update to determine whether the item’s content has changed and whether it should be reindexed.

You should therefore include in the value list the names of all <item> or <entry> child elements whose values change frequently but not as a result of an item’s content change. For example, some content management systems generate feeds dynamically, updating timestamps even if the content remains unchanged.

Example
"MetadataToExcludeFromEtag": {
  "sensitive": false,
  "value": "[\"dynamicElement1\", \"dynamicElement2\"]"
}

UserAgent (String)

This specifies the value of the User-Agent HTTP header that the RSS source uses when requesting the feed. The default value is Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) (compatible; Coveobot/2.0;+http://www.coveo.com/bot.html).

Example
"UserAgent": {
  "sensitive": false,
  "value": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko) (compatible; Coveobot/2.0;+http://www.coveo.com/bot.html)"
}