Edit source extensions

This is for:

Developer System Administrator

An indexing pipeline extension (IPE) is an optional indexing pipeline pre-conversion or post-conversion stage consisting of a Python script that you can apply to one or more sources to customize the way source items are indexed in your Coveo organization. See Indexing pipeline extension overview for details.

Members with the required privileges can add or modify existing indexing pipeline extensions to an organization. As a member of one of these groups, you can then apply an extension to a source using this Edit Source Extensions panel.

Note

You must have the privilege to edit extensions (that is, the Edit access level on the Extensions domain) to copy an extension script. See Extensions domain for details.

An extension can be Common, applying to all item types of a source, or Specific, applying only to specified item types, in which case you must first define the see item types available in your source.

Example

You have a YouTube source that includes videos and playlists. If your indexing pipeline extension applies to both video and playlist items, it’s a common extension. However, if it applies to playlists only, it’s a specific extension.

"Common" tab

The Common tab displays extensions that apply to all source item types.

Access the "Common" tab

On the Sources (platform-ca | platform-eu | platform-au) page, click the source for which you want to manage mappings, and then click More > Manage extensions in the Action bar.

"Common" tab reference

The information you can review in this tab is:

  • Name column: extension name

  • Stage column: stage at which the extension applies, that is, pre-conversion or post-conversion

  • Version column: If you specified an extension versionId to apply in the JSON tab, this version ID is displayed. See Manage other extension versions for details.

"Specific" tab

The Specific tab displays extensions that apply only to certain source item types.

Access the "Specific" tab

  1. On the Sources (platform-ca | platform-eu | platform-au) page, click the source for which you want to manage mappings, and then click More > Manage extensions in the Action bar.

  2. In the Add/Edit Source Extensions panel, select the Specific tab.

"Specific" tab reference

The information you can review in this tab is:

  • Left pane

    • Item type column: item type names.

      Click an item to display the associated item-type-specific extensions in the right pane.

  • Right pane

    • Name column: extension name

    • Stage column : stage at which the extension applies, that is, pre-conversion or post-conversion

    • Version column: If you specified an extension versionId to apply in the JSON tab, this version ID is displayed. See Manage other extension versions for details.

"JSON" tab

The JSON tab shows the source indexing pipeline extension properties in JSON format. Any modification made in the Common and Specific tabs automatically updates the JSON accordingly, and vice versa. You can review this raw source configuration for troubleshooting purposes, but you can also modify the configuration to take advantage of more advanced features not available through the user interface.

The content displayed in the JSON tab is similar to that in the Pre-conversion extensions and Post-conversion extension tabs of the Edit a Source JSON Configuration panel. See Edit a source JSON configuration for details.

Since the JSON tab is so similar to the Edit a Source JSON Configuration panel, the following instructions apply to both:

Access the "JSON" tab

  1. On the Sources (platform-ca | platform-eu | platform-au) page, click the source for which you want to manage mappings, and then click More > Manage extensions in the Action bar.

  2. In the Add/Edit Source Extensions panel, select the JSON tab.

Manage source extensions

The following sections describe individual source extension management actions, but they are often used in a specific sequence. See Apply an extension to a source for details.

Add an item type

Adding an item type in the panel allows you to apply an extension only to items of this specific type, rather than to all source item types.

  1. In the Add/Edit Source Extensions panel, click the Add dropdown menu, and then select Item type.

  2. In the Add an Item Type dialog, enter the Item type name, and then click Add Type.

  3. Back in the Add/Edit Source Extensions panel, do any one of the following:

    Note

    • Clicking Save defers the source rebuild operation, which is necessary to apply changes. Since a source with several millions of items may take weeks to rebuild, choosing the Save option when possible prevents stressing the Coveo and system server resources. It’s therefore recommended to click Save when you plan on making additional changes in the short term, and then to click Save and Rebuild once you’re ready to update your source.

    • Item types with no associated extensions are deleted when you save your changes.

  4. If you clicked Save, back on the Sources (platform-ca | platform-eu | platform-au) page, under the source Status column, click Launch rebuild when you’re ready or wait for the next scheduled source rebuild to make the changes effective.

Apply an extension

The following procedure allows you to apply an existing indexing pipeline extension to your source that was created in your organization before.

  1. In the Add/Edit Source Extensions panel, click the Add dropdown menu, and then select Extension.

  2. In the Apply an Existing Extension on Source Items dialog:

    1. Under Stage, select Pre-conversion or Post-conversion to determine at which stage of the indexing pipeline the extension should apply.

    2. Under Action on Error, select Reject Document to prevent the items from being indexed if there’s an error in the extension, or select Skip Extension if you want the items to be indexed regardless of an error.

    3. Under Apply to, click All items (common) or Specific item types to determine to which items the extension should apply.

    4. (If you selected Specific item types) Select the items types to which the extension should apply. You can also add missing item types.

    5. Under Extension, click the dropdown menu to select an existing extension to apply to your source.

    6. In the Condition(s) to apply box, optionally enter conditions to determine to which items the extension should apply and therefore optimize indexing pipeline performances. See Indexing pipeline extension condition syntax reference for extension condition syntax and examples.

      Example

      Enter %[syssite] == "mycompanyblog.com" AND NOT %[author] == "John Smith" to limit the extension application to mycompanyblog.com, and to exclude articles whose author is John Smith.

    7. Click Apply extension.

  3. In the Add/Edit Source Extensions panel, do any one of the following:

    • If you’re done applying and editing extensions, click Save and Rebuild.

    • If you want to apply or edit other extensions, click Save.

    Note

    Clicking Save defers the source rebuild operation, which is necessary to apply changes. Since a source with several millions of items may take weeks to rebuild, choosing the Save option when possible prevents stressing the Coveo and system server resources. It’s therefore recommended to click Save when you plan on making additional changes in the short term, and then to click Save and Rebuild once you’re ready to update your source.

  4. If you clicked Save, back on the Sources (platform-ca | platform-eu | platform-au) page, under the source Status column, click Launch rebuild when you’re ready or wait for the next scheduled source rebuild to make the changes effective.

Edit an item type name

  1. In the Edit Source Extensions panel, in the Specific tab, click the item type you want to modify, and then click Edit in the Action bar. See Access the "Specific" tab for details.

  2. In the Edit an Extension dialog, edit the item type name, and then click Save.

  3. Back in the Edit Source Extensions panel, do any one of the following:

    • If you’re done adding and editing items and extensions, click Save and Rebuild.

    • If you want to add or edit other items and extensions, click Save.

    Note

    Clicking Save defers the source rebuild operation, which is necessary to apply changes. Since a source with several millions of items may take weeks to rebuild, choosing the Save option when possible prevents stressing the Coveo and system server resources. It’s therefore recommended to click Save when you plan on making additional changes in the short term, and then to click Save and Rebuild once you’re ready to update your source.

  4. If you clicked Save, back on the Sources (platform-ca | platform-eu | platform-au) page, under the source Status column, click Launch rebuild when you’re ready or wait for the next scheduled source rebuild to make the changes effective.

Edit an extension

  1. In the Edit an Extension panel, access the "Common" tab or the "Specific" tab.

  2. In the Common or Specific tab, click the extension you want to modify, and then click Edit in the Action bar.

  3. In the Edit an Extension dialog, edit the extension, and then click Save.

  4. Back in the Add/Edit Source Extensions panel, do any one of the following:

    • If you’re done applying and editing extensions, click Save and Rebuild.

    • If you want to apply or edit other extensions, click Save.

    Note

    Clicking Save defers the source rebuild operation, which is necessary to apply changes. Since a source with several millions of items may take weeks to rebuild, choosing the Save option when possible prevents stressing the Coveo and system server resources. It’s therefore recommended to click Save when you plan on making additional changes in the short term, and then to click Save and Rebuild once you’re ready to update your source.

  5. If you clicked Save, back on the Sources (platform-ca | platform-eu | platform-au) page, under the source Status column, click Launch rebuild when you’re ready or wait for the next scheduled source rebuild to make the changes effective.

Move an extension

  1. In the Edit an Extension panel, access the "Common" tab or the "Specific" tab.

  2. In the Common or the Specific tab, drag and drop extensions using the move icon to rearrange them in the desired execution order. This allows you to set up extensions based on the result of a previously executed extension.

Rebuild source to take account of extension changes

In the Add/Edit Source Extensions panel, click Save and Rebuild to apply your changes.