Manage source mappings

This is for:

Developer System Administrator

Once you’ve created a source and it’s done indexing your content for the first time, you should review and customize the source mappings.

Mappings are necessary for Coveo to index not only your content items, but also important information about them. This will later allow you to configure critical aspects of your Coveo-powered search interface, such as facets.

About mappings

This section explains how mappings work, and how configuring them properly benefits your search interface.

For instructions on how to customize your mappings, jump to Manage mappings.

Item metadata

mappings define the information about your items that Coveo will store in your index. Coveo refers to this information as metadata, but other applications can give it different names, such as properties. In all cases, though, the metadata comes in key-value pairs.

For example, when indexing PDF documents, you’ll likely want to index metadata such as the title, the name of the author, the creation date, and the file size of each document. Typically, the metadata of a PDF is accessible through a properties panel and looks as follows:

Title: Project Planning
Author: John Smith
Created: 06/05/2023 10:34:21 AM
File Size: 988 KB

In this case, Title, Author, Created, and File Size are keys, while Project Planning, John Smith, 06/05/2023 10:34:21 AM, and 988 KB are values. Keys are typically the same for all items of a certain type, but values often differ from one item to another.

Coveo index

Your Coveo index also stores information in key-value pairs. When indexing an item, Coveo takes its metadata values and stores them in containers called fields. Fields may have names similar to your metadata keys, such as title, author, date, and size.

Coveo indexes metadata according to mappings or mapping rules, i.e, instructions that tell Coveo to take the value of a certain key and to store it as the value of a certain field.

For example, when indexing PDF documents, you could want mappings that do the following:

Take value of the …​ key Store it in the …​ field

Title

title

Author

author

Created

date

File Size

size

Automatically and manually created mappings

Depending on the type of content you index, Coveo may create mappings for you. However, you should always review what’s indexed and in which field, and customize your fields and rules as needed.

With native sources, Coveo does the bulk of the work for you by creating default mappings when first indexing your content. For example, with a YouTube source, there’s no need to create fields for the video title, duration, and description, and then to create mappings to populate them with video metadata. Since the metadata keys are always the same for all YouTube sources, Coveo knows what to expect and can do it automatically.

With generic sources the metadata key naming scheme is likely unique to your content system. As a result, Coveo doesn’t know in advance what metadata to expect from your source, and which fields should be populated with this data. You therefore need to manually create fields and mappings to index your content metadata.

However, to help you in this process, with most generic sources (Database, REST API, GraphQL API, Push, and Catalog), if Coveo finds a metadata key that matches an existing field name, it automatically indexes this metadata. For example, if your catalog contains a Price metadata key, and you’ve already added a price field to your organization, Coveo automatically indexes the Price value in the price field.

With any source, however, you’ll need to create or customize mappings when:

  • A metadata key is different from the name of the desired destination field.

  • You want all items to have the same field value. For example, all items in your catalog could have Acme Computer Parts in the manufacturer field.

  • You want to create a custom HTML Quickview for your items. This quickview will be displayed in your search interface.

For more information about the stage of the indexing pipeline where mappings are applied, see Mapping.

Item type

When creating a mapping rule, you can specify one or multiple types of item your rule will apply to. For example, when indexing YouTube content, you could choose to apply a rule to playlists only.

Typically, the item type is indexed as the value of the filetype, documenttype, mappingtype, or objecttype fields. So, before you customize your source mappings, we recommend you use the Content Browser (platform-ca | platform-eu | platform-au) to inspect your source items and note the value that Coveo indexed in these fields.

Then, when customizing mappings, specifying this value will allow you to target all items of this type.

See Source item types for details.

About the performFieldMappingFromAllOrigins parameter

All source JSON configurations contain the performFieldMappingFromAllOrigins parameter. When set to true, it allows Coveo to automatically index metadata if the metadata key in your content matches a Coveo field. This parameter allows you to skip the mapping creation step after you’ve created a source and its fields.

Example

You create a REST API source to index your company directory, which contains phone numbers under metadata key phone. Then, you create a phone field.

Since the performFieldMappingFromAllOrigins parameter is set to true, Coveo automatically indexes the number under phone in the phone field.

Note

Although Coveo automatically indexes the metadata, no mapping rule appears in the Edit the Mappings of a Source panel.

The performFieldMappingFromAllOrigins parameter is especially useful in generic sources. Therefore, it’s set to true by default in Database, REST API, and GraphQL API sources created after January 10, 2024. To set it to true in generic sources created before this date, edit your source JSON configuration, and then launch a source rebuild.

In native sources, the performFieldMappingFromAllOrigins parameter is set to false by default, as Coveo already automatically creates mappings for these sources. We recommend you leave it to false. Setting the parameter to true in native sources comes with the risk of indexing irrelevant metadata and cluttering your index, which leads to a decrease in performance. This may also lead to metadata being indexed in fields where it’s not relevant, which in turn pollutes your facets with unwanted values.

Taking advantage of the performFieldMappingFromAllOrigins parameter to automatically index metadata may appear as a time-saving strategy. However, if you do so, keep in mind that, to build search interfaces that offer an optimal experience, you must keep your fields and the information they contain clean and organized. In other words, make sure you don’t have redundant fields and that the information indexed in each field is adequate.

For example, let’s say you already have a jobtitle field in your organization, and you want to index a Database source in which job titles are saved under the metadata key work_title. You should definitely create a mapping rule to index the work_title metadata in the jobtitle field. If you were to create a work_title field to take advantage of automatic metadata indexing, you’d end up with a source populating the jobtitle field and another source indexing similar information in the work_title field. Then, when creating a facet to display in your search interface, you’d have to choose either of these fields. As a result, the facet would show only a part of the job titles indexed by Coveo, and your search interface users wouldn’t be able to filter your content optimally.

Similarly, let’s say you already have the title field in your organization. This field is used to index titles of PDF documents. When creating a Database source to index an employee directory, you look at the content metadata and notice that job titles appear under the title metadata key. With the performFieldMappingFromAllOrigins parameter set to true, if you don’t create a mapping rule, job titles will be automatically indexed in the title field. A facet based on the title field would therefore show both job titles and document titles. To avoid this, you must create a mapping rule to ensure that data under title is indexed somewhere else.

The mappings panel is accessible from the Sources (platform-ca | platform-eu | platform-au) page of the Coveo Administration Console. Click the source of which you want to manage the mappings, and then click More > Manage mappings in the Action bar.

Alternatively, you can also access the panel from the View Metadata subpage.

The panel consists of three tabs: Common, Specific and JSON.

"Common" and "Specific" tabs

The Common and the Specific tabs work in a similar way. They show a list of fields and the corresponding mapping rule, that is, what Coveo will store in each field. Both tabs also display information about the fields, that is, their type and whether some options are enabled.

The difference between the Common and the Specific tab is the items that the displayed rules apply to. The Common tab shows rules that apply to all items of a source, while the Specific tab displays rules that apply to items of a specific type only. When you select an item type on the left, the right part of the tab displays the corresponding fields and mapping rules.

For example, when indexing YouTube videos and playlists from your corporate channel, you could decide to apply a mapping rule to the playlists only. This rule would appear in the Specific tab, when you’d click playlist.

Playlist mapping rule | Coveo

"JSON" tab

The JSON tab shows the source mapping in JSON format. Any modification made in the Common and Specific tabs automatically updates the JSON accordingly, and vice versa.

You can also review the mappings of a source in the Mappings tab of the Edit a Source JSON Configuration panel. Although the information is structured differently, the following instructions apply to both panels:

Manage mappings

Once your source is done indexing your content for the first time, you can customize its mappings.

Look at the metadata that Coveo might already have indexed, either in the View Metadata subpage or in the Content Browser (platform-ca | platform-eu | platform-au). Think of the information that your search interface end users will find relevant when searching your content and of the facets you intend to add to your search interface. Ask yourself:

  1. Has all the relevant metadata been indexed?

  2. Is the metadata indexed in the appropriate fields?

If the answer is no, you must customize your source mappings.

  1. Determine whether you want to apply your mapping to some or all items of your source. Check the Content Browser (platform-ca | platform-eu | platform-au) for the properties of the items indexed by your source. The item type typically appears in the filetype, documenttype, mappingtype, or objecttype fields.

  2. If you want to apply the mapping to some items only, add the desired item types.

    Click for item type addition procedure

    1. In the Edit the Mappings of a Source panel, click the Add dropdown menu, and then select Item type.

    2. In the Add an Item Type dialog, click the dropdown menu, and the select or create an item type. Then, click Add Type.

  3. In the Edit the Mappings of a Source panel, select the desired tab, depending on the mapping to add/edit.

  4. To add a mapping, click the Add dropdown menu, and then select Mapping. To edit a mapping, click the desired rule, and then click Edit in the Action bar.

  5. Either way, in the panel that opens:

    1. Under Field, select the field where Coveo will index your content metadata. You can also a add or edit a field if needed.

    2. Under Apply to, click All items (common) or Specific item types to determine to which items the mapping should apply.

    3. Under Rules, enter the mapping rule to apply. See Mapping rule syntax reference if needed. If you enter more than one rule, Coveo will evaluate them in the order they appear, until one of them yields a non-empty value.

    4. Click Apply mapping.

  6. Back in the Edit the Mappings of a Source panel, click Save and rebuild source if you’re done making changes. Coveo will launch a source rebuild, which is necessary to apply your changes.

    Tip

    Alternatively, click Save to close the panel without launching a rebuild. This option is especially useful when you plan on making additional changes in the short term. Since a source with several millions of items may take weeks to rebuild, you will save time and resources by rebuilding it only when necessary.

Required privileges

You can assign privileges to allow access to specific tools in the Coveo Administration Console. The following table indicates the privileges required to view or edit elements of the Sources (platform-ca | platform-eu | platform-au) page and associated panels. See Manage privileges and Privilege reference for more information.

Actions Service Domain Required access level

View sources, view source update schedules, and subscribe to source notifications

Content

Fields

View

Sources

Organization

Organization

Edit sources, edit source update schedules, and view the View Metadata subpage

Content

Fields

Edit

Sources

Content

Source metadata

View

Organization

Organization

What’s next?

Once your source is rebuilt, ensure that your mappings work as expected in the Content Browser. In the Fields tab of your item properties, check that the desired metadata values have been properly indexed. Adjust your mapping rules if necessary.