Manage source mappings

This is for:

Developer System Administrator

Once you’ve created a source and indexed your content for the first time, you should review and customize the source mappings.

The Coveo Platform uses these mappings to index important information about your content items. After you’ve defined the mappings, you’ll be able to configure critical aspects of your Coveo-powered search interface, such as facets.

About mappings

This section explains how mappings work, and how configuring them properly benefits your search interface.

For instructions on how to customize your mappings, jump to Manage mappings.

Item metadata

mappings define the information about your items that Coveo will store in your index. Coveo refers to this information as metadata, but other applications can give it different names, such as properties. In all cases, though, the metadata comes in key-value pairs.

For example, when indexing PDF documents, you’ll likely want to index metadata such as the title, author, creation date, and file size of each document. Typically, the metadata of a PDF is accessible through a properties panel and looks as follows:

Title: Project Planning
Author: John Smith
Created: 06/05/2023 10:34:21 AM
File Size: 988 KB

In this case, Title, Author, Created, and File Size are keys, while Project Planning, John Smith, 06/05/2023 10:34:21 AM, and 988 KB are values. Keys are typically the same for all items of a certain type, but values often differ from one item to another.

Coveo index

Your Coveo index also stores information in key-value pairs. When indexing an item, Coveo takes its metadata values and stores them in containers called fields. Fields may have names similar to your metadata keys, such as title, author, date, and size.

Coveo indexes metadata according to mappings or mapping rules, i.e, instructions that tell Coveo to take the value of a certain key and to store it as the value of a certain field.

For example, when indexing PDF documents, you might want mappings that do the following:

Take value of the …​ key Store it in the …​ field

Title

title

Author

author

Created

date

File Size

size

Default fields and mappings

Depending on the type of content you index, Coveo may create fields and mappings for you. However, you should always review what’s indexed and in which field. You can customize your fields and rules as needed.

With native sources, Coveo does the bulk of the work for you by creating default fields and mappings when first indexing your content. For example, with a YouTube source, there’s no need to create fields for the video title, duration, and description, and then to create mappings to populate them with video metadata. Since the metadata keys are always the same for all YouTube sources, Coveo knows what to expect and can do it automatically.

With generic sources, the metadata key naming scheme is likely unique to your content system. As a result, Coveo doesn’t know in advance what metadata to expect from your source, and which fields should be populated with this data. You therefore need to manually create fields to store your content metadata. You can create mappings manually, or you can use automatic mapping.

With any source, however, you’ll need to create or customize mappings when:

  • A metadata key is different from the name of the desired destination field.

  • You want all items to have the same field value. For example, all items could have Acme Computer Parts in the manufacturer field.

  • You want to create a custom HTML Quickview for your items. This quickview will be displayed in your search interface.

For more information about the stage of the indexing pipeline where mappings are applied, see Mapping.

Item type

When creating a mapping rule, you can specify one or multiple types of item your rule will apply to. For example, when indexing YouTube content, you could choose to apply a rule to playlists only.

Typically, the item type is indexed as the value of the filetype, documenttype, mappingtype, or objecttype fields. So, before you customize your source mappings, use the Content Browser (platform-ca | platform-eu | platform-au) to inspect your source items and note the value that Coveo indexed in these fields.

Then, when customizing mappings, specifying this value will allow you to target all items of this type.

See Source item types for details.

Automatic mapping

Every source JSON configuration includes the performFieldMappingUsingAllOrigins setting at the root level, under the All tab. When set to true, Coveo automatically maps metadata to any existing field with a matching name, eliminating the need to create source mappings manually. This is called automatic mapping.

The performFieldMappingUsingAllOrigins setting is especially useful in generic sources and is set to true by default in Push and Catalog sources, as well as in Database, REST API, and GraphQL API sources created after January 10, 2024. To set it to true in sources created before this date, edit your source JSON configuration, and then launch a source rebuild.

Example

You create a REST API source to index your company directory, which contains phone numbers under metadata key phone. Then, you create a phone field.

Since the performFieldMappingUsingAllOrigins setting is set to true, Coveo automatically indexes the number under phone in the phone field.

However, any source mappings you configure take precedence over automatic mapping for the specified metadata. The following diagram illustrates how the Coveo indexing pipeline handles metadata mapping when performFieldMappingUsingAllOrigins is enabled:

Metadata auto-mapping flow

Note

Although Coveo automatically indexes the metadata, no mapping rule appears on the Edit mappings subpage.

In native sources, the performFieldMappingUsingAllOrigins setting is set to false by default, as Coveo already automatically creates default fields and mappings for these sources. It’s safer to leave it to false. Setting performFieldMappingUsingAllOrigins to true in native sources comes with the risk of indexing irrelevant metadata and cluttering your index, which leads to a decrease in performance. This may also lead to metadata being indexed in fields where it’s not relevant, which in turn pollutes your facets with unwanted values.

Taking advantage of the performFieldMappingUsingAllOrigins setting to automatically index metadata may appear as a time-saving strategy. However, if you do so, keep in mind that, to build search interfaces that offer an optimal experience, you must keep your fields and the information they contain clean and organized. In other words, make sure you don’t have redundant fields and that the information indexed in each field is adequate.

For example, let’s say you already have a jobtitle field in your organization, and you want to index a Database source in which job titles are saved under the metadata key work_title. You should definitely create a mapping rule to index the work_title metadata in the jobtitle field. If you were to create a work_title field to take advantage of automatic metadata indexing, you would end up with a source populating the jobtitle field and another source indexing similar information in the work_title field. Then, when creating a facet to display in your search interface, you would have to choose either of these fields. As a result, the facet would show only a part of the job titles indexed by Coveo, and your search interface users wouldn’t be able to filter your content optimally.

Similarly, let’s say you already have the title field in your organization. This field is used to index titles of PDF documents. When creating a Database source to index an employee directory, you look at the content metadata and notice that job titles appear under the title metadata key. With performFieldMappingUsingAllOrigins set to true, if you don’t create a mapping rule, job titles will be automatically indexed in the title field. A facet based on the title field would therefore show both job titles and document titles. To avoid this, create a mapping rule to ensure that data under title is indexed somewhere else.

The Edit mappings subpage is accessible from the Sources (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console. Click the source of which you want to manage the mappings, and then click Mappings in the Action bar.

The subpage has a Common tab and a Specific tab.

"Common" and "Specific" tabs

The difference between the Common and Specific tabs is the items the displayed mapping rules apply to. The Common tab shows a list of rules that apply to all items of a source, while the Specific tab displays rules that apply to items of a specific type only.

For example, when indexing YouTube videos and playlists from your corporate channel, you could decide to apply a mapping rule to the playlists only. The following shows a specific mapping rule created to populate the body field for playlist items, effectively overriding the common source mapping for the body field.

Playlist mapping rule | Coveo

On the Specific tab, item types are listed with their number of mapping rules and the mapping Status.

Note

An item type must have at least one mapping rule. If an item type has no mapping rule, its status will say No associated mappings. Will be deleted on rebuild.

Whether you’re on the Common tab or you’re viewing the mappings for a specific item type, the subpage shows a list of fields and the corresponding mapping rule, that is, what Coveo will store in each field. The subpage also displays information about the fields, that is, their type and whether some options are enabled.

"Edit with JSON" button

Click the Edit with JSON button to open the source mapping in JSON format. Any modification made in the Common and Specific tabs automatically updates the JSON accordingly, and vice versa.

You can also review the mappings of a source on the Mappings tab of the Edit configuration with JSON panel. Although the information is structured differently, the following instructions apply to both configuration screens:

Manage mappings

Once your source is done indexing your content for the first time, you can customize its mappings.

Look at the metadata that Coveo might already have indexed, either on the View and map metadata subpage or in the Content Browser (platform-ca | platform-eu | platform-au). Think of the information that your search interface end users will find relevant when searching your content and of the facets you intend to add to your search interface. Ask yourself:

  1. Has all the relevant metadata been indexed?

  2. Is the metadata indexed in the appropriate fields?

If the answer is no, you must customize your source mappings.

Important

If you use a Catalog source to index your catalog data, don’t define source mappings for the standard commerce fields.

The standard field mappings defined in the catalog configuration will overwrite any source mappings for the standard commerce fields.

  1. Determine whether you want to apply your mapping to some or all items of your source. Check the Content Browser (platform-ca | platform-eu | platform-au) for the properties of the items indexed by your source. The item type typically appears in the filetype, documenttype, mappingtype, or objecttype fields.

  2. Choose the option that matches your case:

    To add a common mapping

    1. On the Common tab of the Edit mappings subpage, click Add mapping at the top right.

    2. In the Add a common mapping modal, do the following:

      1. Under Field, select the field where Coveo will index your content metadata. You can also add a field if needed.

      2. Under Rules, enter the mapping rule to apply. See Mapping rule syntax reference if needed. If you enter more than one rule, Coveo will evaluate them in the order they appear, until one of them yields a non-empty value.

      3. Click Add mapping.

    To edit a common mapping

    1. On the Common tab of the Edit mappings subpage, select the field whose mapping you want to edit, and then click Edit mapping in the Action bar.

    2. In the Edit common mapping modal, do the following:

      1. Under Field, change the field where Coveo will index your content metadata, if necessary. You can also add a field if needed.

      2. Under Rules, update the mapping rule to apply for the selected field, if necessary. See Mapping rule syntax reference if needed. If you configure more than one rule, Coveo will evaluate them in the order they appear, until one of them yields a non-empty value.

      3. Click Apply mapping.

    To add a mapping to an existing item type

    1. On the Specific tab of the Edit mappings subpage, select the item type, and then click Add mapping in the Action bar.

    2. In the Add a specific mapping that applies to this item type modal, do the following:

      1. Under Field, select the field where Coveo will index your content metadata. You can also add a field if needed.

      2. Under Rules, enter the mapping rule to apply. See Mapping rule syntax reference if needed. If you enter more than one rule, Coveo will evaluate them in the order they appear, until one of them yields a non-empty value.

      3. Click Add mapping.

    To add a mapping to multiple existing item types

    1. On the Specific tab of the Edit mappings subpage, click Add mapping at the top right.

    2. In the Add a specific mapping modal, do the following:

      1. Under Field, select the field where Coveo will index your content metadata. You can also add a field if needed.

      2. Under Select the applicable item types, select the item types.

      3. Under Rules, enter the mapping rule to apply. See Mapping rule syntax reference if needed. If you enter more than one rule, Coveo will evaluate them in the order they appear, until one of them yields a non-empty value.

      4. Click Add mapping.

    To edit a mapping for a given item type

    1. On the Specific tab of the Edit mappings subpage, select the item type, and then click View mappings in the Action bar.

    2. Select the field whose mapping you want to edit, and then click Edit mapping in the Action bar.

    3. In the Edit a specific mapping that applies to this item type modal, do the following:

      1. Under Field, change the field where Coveo will index your content metadata, if necessary. You can also add a field if needed.

      2. Under Rules, update the mapping rule to apply for the selected field, if necessary. See Mapping rule syntax reference if needed. If you configure more than one rule, Coveo will evaluate them in the order they appear, until one of them yields a non-empty value.

      3. Click Apply mapping.

    To add an item type and its first mapping

    1. On the Specific tab of the Edit mappings subpage, click Add item type at the top right.

    2. In the Add an item type modal, do the following:

      1. Under Item type name, enter the name of the item type.

      2. Under Field, select the field where Coveo will index your content metadata. You can also add a field if needed.

      3. Under Rules, enter the mapping rule to apply. An item type can’t be created without a mapping rule. See Mapping rule syntax reference if needed. If you enter more than one rule, Coveo will evaluate them in the order they appear, until one of them yields a non-empty value.

      4. Click Add item type and mapping.

        Tip

        To rename an existing item type later, select it on the Specific tab, and then click Edit item type in the Action bar.

  3. When you’re done making changes, click Save and rebuild source. Coveo launches a source rebuild, which is necessary to apply your changes. Alternatively, click Save to keep your changes without rebuilding. This is useful when you want to make more mapping changes before the rebuild.

Required privileges

The following table indicates the privileges required to view or edit elements of the Sources (platform-ca | platform-eu | platform-au) page and associated panels. See Manage privileges and Privilege reference for more information.

Actions Service Domain Required access level

View sources, view source update schedules, and subscribe to source notifications

Content

Fields

View

Sources

View

Organization

Organization

View

Edit sources, edit source update schedules, and edit source mappings

Content

Fields

Edit

Sources

Edit

Organization

Organization

View

View and map metadata

Content

Fields

View

Source metadata

View

Sources

Edit

Organization

Organization

View

What’s next?

Once your source is rebuilt, ensure that your mappings work as expected in the Content Browser. In the Fields tab of your item properties, check that the metadata values have been properly indexed. Adjust your mapping rules if necessary.