Add a Push source

This is for:

Developer System Administrator

Coveo has dedicated connectors for many web and on-premises systems, letting you quickly make application content searchable. See the connector directory for the full list.

However, you may want to index the content of applications that don’t have a dedicated connector. In such cases, the Push source is one of the generic sources to consider for making the desired content searchable with Coveo.

For example, let’s say you have an on-premises content management system (CMS) developed in-house. With a push approach, one of your developers could create a crawler to retrieve this content and push it to your Coveo organization using the Push API. As the name suggests, this indexing approach involves only outbound requests from your infrastructure, with no inbound requests from the Coveo Platform. But before pushing content to your Coveo organization, you must create a Push source to receive the content.

Architecture diagram of Coveo’s push indexing components

The following diagram shows the main interactions between a custom crawler, the Push API, the indexing pipeline, a Push source, and the other components involved in this process.

Diagram showing Push API interactions | Coveo

Source key characteristics

The following table presents the key characteristics of a Push source.

Features Supported Additional information

Content update operations

All content update operations are triggered by Push API requests.

Content security options

Same users and groups as in your content system

check

Specific users and groups

check

Everyone

check

metadata indexing for search

Automapping of metadata to a field with a matching name

Enabled by default.

Automatically indexed metadata

Sample of autopopulated fields (no user-defined metadata required):

Sample of autopopulated fields (API request parameter of matching name required):

  • author

  • title
     

After adding or updating items, inspect your item field values in the Content Browser.

Collected indexable metadata

Parameters specified in API request payloads that don’t match an existing field name are collected, but not indexed.

After adding or updating items, review the View and map metadata subpage for the list of indexed metadata and to index additional metadata from those available.

Add a Push source

Follow the instructions below to add a Push source.

  1. On the Sources (platform-ca | platform-eu | platform-au) page, click Add source.

  2. In the Add a source of content panel, click the Push source tile.

  3. Configure your source.

Tip
Leading practice

It’s best to create or edit your source in your sandbox organization first. Once you’ve confirmed that it indexes the desired content, you can copy your source configuration to your production organization, either with a snapshot or manually.

See About non-production organizations for more information and best practices regarding sandbox organizations.

"Configuration" tab

On the Add a Push source page, the Configuration tab is selected by default. It contains your source’s general information and push process options.

General information

Name

Enter a name for your source.

Tip
Leading practice

A source name can’t be modified once it’s saved, therefore be sure to use a short and descriptive name, using letters, numbers, hyphens (-), and underscores (_). Avoid spaces and other special characters.

When coding a crawler to push content in this source, a developer will need this name to get the corresponding sourceId from the Sources (platform-ca | platform-eu | platform-au) page.

Project

Use the Project selector to associate your source with one or more Coveo projects.

"Push process" section

API key

Select the Create an API key checkbox to create an API key with the privileges required to push items to the source and, if the Same users and groups as in your current permission system Content security option is selected, those required to push security identities to the relevant security identity provider.

Privilege Everyone content security Same users and groups as in your current permission system content security

Content > Push items to sources > Custom (Allow on target source)

check

check

Content > Sources > View all

check

check

Organization > Organization > View

check

check

Content > Push identities to security providers > Allow for all providers

check

Content > Security identity providers > View

check

The API key option is only visible when the source is being created. The option is enabled provided that you have the privilege to edit API keys as well as the privileges you need to grant on the API key.

If the Push source you’re creating has the Same users and groups as in your current permission system Content security option selected, an error occurs if the following conditions are met:

  • The Create an API key checkbox is selected.

  • You don’t have the necessary content security-related privileges.

To create an API key for the source

  1. If you want the search interface to replicate the permission system of the indexed content repository, select the Same users and groups as in your current permission system option in the Content security tab.

  2. Select the Create an API key checkbox.

  3. Resume source configurations or click Add source to create the source immediately.

The API key successfully created dialog containing your private key value will appear after you click Add source. Click clipboard-text to copy the key value to your clipboard.

Note

If you don’t create an API key at source creation, you can do so later from the Sources (platform-ca | platform-eu | platform-au) page.

Creating a Push source API key | Coveo

When creating an API key from the Sources (platform-ca | platform-eu | platform-au) page, the assigned privileges are the same as those listed in the table above.
Push API URL

This field, available only after source creation, contains the URL you must use to add and update an item in your source in the following format:

Primay region URL

US

https://api.cloud.coveo.com/push/v1/organizations/<ORGANIZATION_ID>/sources/<SOURCE_ID>/documents

Canada

https://api-ca.cloud.coveo.com/push/v1/organizations/<ORGANIZATION_ID>/sources/<SOURCE_ID>/documents

Europe

https://api-eu.cloud.coveo.com/push/v1/organizations/<ORGANIZATION_ID>/sources/<SOURCE_ID>/documents

Australia

https://api-au.cloud.coveo.com/push/v1/organizations/<ORGANIZATION_ID>/sources/<SOURCE_ID>/documents

The URL contains your organization ID and source ID, which are required parameters for all operations on your source.

Click copy16px to copy the URL to your clipboard.

"Items" tab

In the Items tab, you can enable or disable optical character recognition (OCR) on your content.

Content and images

If you want Coveo to extract text from image files or PDF files containing images, enable the appropriate option. The extracted text is processed as item data, meaning that it’s fully searchable and will appear in the item Quick view.

Note

When OCR is enabled, ensure the source’s relevant file type configurations index the item content. Indexing the item’s metadata only or ignoring the item will prevent OCR from being applied.

See Enable optical character recognition for details on this feature.

"Content security" tab

Select who will be able to access the source items through a Coveo-powered search interface. For details on this parameter, see Content security.

Note

If you make your source secure by selecting the Same users and groups as in your current permission system content security option, your crawler process must push item permissions along with the content (see Manage security identities in a security identity provider and Manage items and permissions in a Push source).

"Access" tab

In the Access tab, specify whether each group (and API key, if applicable) in your Coveo organization can view or edit the current source.

For example, when creating a new source, you could decide that members of Group A can edit its configuration, while Group B can only view it.

For more information, see Custom access level.

Required privileges

Actions Service Domain Required access level

View sources, view source update schedules, and subscribe to source notifications

Content

Fields

View

Sources

Organization

Organization

Edit sources, edit source update schedules, and edit source mappings

Organization

Organization

View

Content

Fields

Edit

Sources

Manage Push source items, update the Push source status

Content

Push items to sources

Allow (on target source)

Sources

View

Organization

Organization

View

Manage security identities

Content

Push identities to security providers

Allow (on target providers)

Security identity providers

View

Organization

Organization

View

View and map metadata

Content

Source metadata

View

Fields

Organization

Organization

Content

Sources

Edit
Notes

  • The Edit all privilege isn’t required to create sources. When granting privileges for the Sources domain, you can grant a group or API key the View all or Custom access level, instead of Edit all, and then select the Can Create checkbox to allow users to create sources. See Can Create ability dependence for more information.

  • The ability to manage security identities is only required when using the Same users and groups as in your current permission system content security option.

What’s next?

  1. If your Push source is secured, create a security identity provider.

  2. Create the fields you need in your index.

    Remember, any metadata you push to your source is automatically mapped to an existing field with the same name. For example, if your Coveo organization has a version field, version metadata values in the items you push to your source will automatically populate this field.

    Important

  3. Use the Push API to push content to your source.