Adding and Managing Sources

A source is a virtual container holding all items related to a specific system (Twitter®, Web, SharePoint®…). Specific data from items within the source is collected to form an index. Coveo Cloud connects to a specific system, extracts its data content to create a source, indexes that source, and maintains the source up to date. Sources can be made part of the scope of one or more search interfaces to make its content searchable by end users that have appropriate permissions.

Use the Sources page to initially create your sources, and come back anytime to manage them.

Page Accessibility

Access the “Sources” Page

1. If not already done, log in to the Coveo Cloud platform as a member of a group with the required privileges to manage sources in the target Coveo Cloud organization (see Required Privileges).

2. In the main menu on the left, under Content, select Sources.

Access the Mapping Management Panel

Mappings define the content of Coveo index fields for each source item (see Adding and Managing Source Mappings).

1. In the Sources page, in the source list, select the source whose mappings you want to manage (see Access the Sources Page).

2. In the Action bar, click More and select Manage mappings.

Access the Extension Management Panel

In the Edit Source Extensions: [SourceName] panel, you can apply indexing pipeline extensions to a source (see Adding and Managing Extensions and Edit Source Extensions).

1. In the Sources page, in the source list, select the source whose extensions you want to manage (see Access the Sources Page).

2. In the Action bar, click More, and then select Manage extensions (see Manage Source Extensions).

Add, Edit, Duplicate, and Delete Sources

Add a New Source

1. In the Sources page, click Add Source (see Access the Sources Page).

   If the Add Source button is grayed and unresponsive, you do not have all required privileges to perform this action (see Required Privileges).

2. In the panel that appears, select the type of source that you want to create.

   Sources labeled Unavailable are not allowed by your Coveo Cloud V2 license. Contact Coveo to upgrade your license.

3. Refer to the appropriate article for details on how to create each source (see Available Coveo Cloud V2 Connectors).

   Plan and pick source names carefully. Once created, source names can only be changed by completely deleting and rebuilding the associated source, a cumbersome task, especially with large sources that require several hours to rebuild.

Duplicate a Source

If you intend to create two sources with a similar configuration, you should consider creating a source, and then duplicating it. The source duplication feature can save you time, especially if the source configuration involves more than a few parameters. Once your first source is duplicated, you can then edit the desired parameters in the source copy­.

1. In the Sources page, select the source you want to duplicate (see Access the “Sources” Page).
2. In the Action bar, click More, and the select Duplicate.
3. In the Duplicate a Source: [Source Name] panel that appears, enter a Source name for the copy, and then click Duplicate source.
4. Back in the Sources page, once your new source has been added, edit it to make the desired changes (see Edit an Existing Source).
5. Build your new source (see Refresh, Rescan, or Rebuild Sources).

Edit an Existing Source

Once a source is created, you can always come back and edit its configuration.

1. In the Sources page, in the source list (see Access the Sources Page):

   • Click the source that you want to edit, and then in the Action bar, click Edit.

     OR

   • Double-click the source that you want to edit.

   Grayed-out sources are sources for which you only have the View access level (see Understanding the Custom Access Level). You cannot edit these sources, but you can click View in the Action bar to review their configuration.

2. In the Edit a [SourceType] Source: [SourceName] panel, make the desired changes, and then depending on the changes you made:

   • When the changes impact source items (e.g., you changed the source URL) or you want the changes to be effective as soon as possible, click Save and Rebuild.

     OR

   • When your changes does not affect source items (e.g., you updated source credentials) or you want to ensure your changes are correct before applying them, click Save.

   • Clicking Save defers the source rebuild operation, which is necessary to apply changes to the indexed items. Therefore, if you made changes that will affect the indexed items, it is recommended to rather use the Save and Rebuild option (see Save and Rebuild). However, if you are sure that your changes will not affect the indexed items in any way, such as changes relative to the source credentials, you may click Save to avoid an irrelevant rebuild operation. Since a source with several millions of items may take weeks to rebuild, choosing the Save option when possible prevents stressing the Coveo Cloud and system server resources.

   • If you change fields or mappings and do not rebuild the source, the changes will apply only to new source items following a source refresh.

3. If you clicked Save, back in the Sources page, under the source Status column, click Start required rebuild when you are ready or wait for the next scheduled source rebuild to make the changes effective.

Edit the Source Configuration in JSON Format

You can view and change the source configuration in JSON format. That format lists all the source parameters in key-value pairs, including the mappings (see Edit a Source JSON Configuration).

Delete an Existing Source

Deleting a source permanently erases the source configuration and its indexed content.

1. In the Sources page, in the source list, select the source that you want to delete (see Access the Sources Page).

2. In the Action bar, click More and select Delete.

3. At the Are you sure? prompt, click Delete.

Update Access Token

OAuth sources such as Salesforce and Google Drive use access tokens that need to be updated manually when expired. You can easily update an OAuth source access token.

1. In the Sources page, in the source list, select the OAuth source for which you want to update the access token (see Access the Sources Page).

2. In the Action bar, click More, and then select Update access token.

3. In the Update Access Token confirmation dialog:

   1. (For secured OAuth sources only) Optionally, click the Also update the token in the source identity security provider configuration check box.

      Ensure no source configurations will become invalid before updating the access token used by the source identity security provider.

   2. Click Update.

4. In the page that appears, enter the email address and password of the account that you created for the source to index your content, and then authorize update by clicking Authorize, Allow, or Log In, depending on the source system.

   By doing so, you grant Coveo Cloud read right to your source content using OAuth, a protocol that grants access to external applications without exposing the user’s real credentials. You can always revoke the Coveo Cloud organization OAuth authorization (e.g., see Manage OAuth Access for Your Connected Apps for Salesforce accounts and Revoke third-party access to your account information for Google accounts).

Manage Source Content Updates

Refresh, Rescan or Rebuild Sources

While the best practice is to configure well thought out source update schedules (see Schedule Source Updates), you can also start a source update (refresh, rescan, or rebuild) manually. This is useful when you want to ensure that source content is up to date following a specific change in the indexed system.

1. In the Sources page, in the source list, select the source that you want to update (see Access the Sources Page).

2. In the Action bar, click Refresh, Rescan, or Rebuild.

   OR

   Click More, and select Refresh, Rescan, or Rebuild depending on what you want to achieve (see Refresh VS Rescan VS Rebuild).

   • Not all connectors support refresh.

   • Changes on items already processed by an ongoing source update are taken into account by the next scheduled update.

   • Source operations retrieve the latest version of unprocessed items.

     You start a rescan at 12:00 AM with a source of 100 items. At 12:01 AM, when the rescan processes the 50th item, the title of the 81st item changes. The rescan takes the change into account and thus updates the 81st item title.

3. Back in the source list, in the Status column for the selected source, monitor the progress for the operation that you selected.

   You can also get more update action information from the Activity panel (see Review the Activity of Individual Sources).

4. You can pause or cancel (rescan and rebuild only) an ongoing source update by selecting the source row > More menu > Pause/Cancel [Update_Operation].

   • Source configuration changes (e.g., mappings, fields) that occur when a source update is paused are taken account when the source operation is resumed.

     During a paused source rebuild, you add a mapping rule in the related source configuration. You resume the paused rebuild, which will take your new mapping rule into account.

   • Not all sources support the pause and cancel operations.

   • A paused source operation will get cancelled after a period of inactivity. The duration of that period depends on the reason why the source operation was paused. An operation can either be paused manually or paused due to an issue in the source or Coveo Platform in which case the source status will be Paused on error.

Schedule Source Updates

To edit a source update schedule, see Edit a Source Schedule.

Resume a Paused Source Update

When a source update is in the paused state, a button appears in the Action bar.

1. In the Sources page, in the source list, select the paused source for which you want to resume the update procedure (refresh, rebuild, or rescan) (see Access the Sources Page).

2. In the Action bar, click Resume refresh, Resume rebuild, or Resume rescan, depending on the paused update.

Inspect Sources

Review the Status and Size of All Your Sources

1. On the Sources page, in the table, review the content in the Status column to determine the current source state as well as details on recent or upcoming activity (see Access the Sources Page).

2. In the table, review the number and size of source items in the Content column.

Review the Activity of Individual Sources

1. In the Sources page, in the table, select the source whose activity you want to review (see Access the Sources Page).

2. In the Action bar, click Activity.

   If the Activity option is missing, you do not have all required privileges to perform this action (see Required Privileges).

3. Review the Activity panel (see Review Events Related to Specific Coveo Cloud Administration Console Resources).

Browse the Content of a Source

You can easily review the content of a source using the Content Browser (see Inspect Items With the Content Browser).

1. In the Sources page, in the source list, select the source that you want to inspect (see Access the Sources Page).

2. In the Action bar, click More, and select Open in Content Browser.

   The Content Browser page opens, and the source you selected is automatically selected in the Source facet so that only associated items appear in the search results.

3. Use the Content Browser features to inspect your source content (see Inspect Items With the Content Browser).

Browse the Logs of a Source

You can review the log entries of a source using the Log Browser (see Review Item Logs).

1. In the Sources page, in the source list, select the source of which you want to inspect the logs (see Access the Sources Page).

2. In the Action bar, click More, and select Open in Log Browser.

   The Log Browser page opens up, and the source you selected is automatically selected in the Source facet so that only associated log entries appear in the search results.

3. Use the Log Browser features to inspect your source logs (see Review Item Logs).

Manage Source Notification Subscriptions

You can manage source notification subscriptions to specify whether you want to receive email notifications when certain activities are triggered by a specific source (see User Notifications).

1. In the Sources page, in the source list, select the source you want to receive, or cease receiving email notifications for (see Access the Sources Page).
2. In the Action bar, click More and select Subscribe to notifications (or Edit subscription to notifications) to access the subscription management panel, or select Unsubscribe from notifications to cease receiving notifications for that source.

Navigate the “Sources” page

Navigate Through Source Pages

(When your organization contains more than 50 sources) In the Sources page, at the bottom-right of the table, click the left and right arrow icons, or a page number to navigate through pages (see Access the Sources Page).

Filter Visible Table Elements

When there are numerous sources listed in the page and you want to find one or more specific sources more easily:

1. In the Sources page, in the filter box on the Action bar, type characters corresponding to textual elements that you are looking for in the table (see Access the Sources Page).

   Sources with no matching textual elements disappear, leaving only those that match. The filter matches any text string entered in the table.

   In the filter box:

   To only show sources	Start typing
   Of type Salesforce	salesforce
   Of type Cloud	cloud
   With the security type Shared	shared
   With web in the Name	web
   Where Content is expressed in GB	gb

2. Clear the filter box content to list all sources again.

Set the Number of Sources per Page

(When your organization contains more than 50 sources) In the Sources page, at the bottom-left of the page, select 25, 50, or 100 (see Access the Sources Page).