THIS IS ARCHIVED DOCUMENTATION

Understanding Sitecore, Coveo for Sitecore, and Coveo Fields

This is for:

Developer System Administrator

A Sitecore item generally inherits from a template. A template defines a set of fields, which can be of any one of these types: Single-Line Text, Multi-Line Text, Number, Date, Droplist, Droplink, etc. Therefore, in a sense, a Sitecore item is an aggregation of fields.

A Coveo item is also composed of several fields, which can be of any one of these types: String, Integer, Floating point, and Date/time. Therefore, a Coveo item is also an aggregation of fields.

The role of Coveo for Sitecore is to index Sitecore items along with their fields in a way that’s coherent with the Coveo index. It:

  1. Converts Sitecore field values to a format compatible with the Coveo index.

  2. Adds a field mapping to the Coveo source, which is used to map Sitecore fields with Coveo fields when indexing.

Coveo fields are most commonly divided into custom and system fields. Coveo already defines several system fields, which for the most part are filled automatically by Coveo: title, author, uri, etc. However, custom fields aren’t handled in the same way. As soon as you rebuild your search indexes, Coveo for Sitecore gets all the Sitecore fields that you have added to the Included Fields list and, for each of these fields, one custom Coveo field is created.

The role of Coveo for Sitecore is to index Sitecore items along with their fields in a way that’s coherent with the Coveo index. It:

  1. Translates Sitecore field names to Coveo field names, while still retaining from which Sitecore database the fields come from. This is necessary, as field names must be unique in a Coveo index, whereas Sitecore doesn’t prevent duplicate fields (see Handling Duplicate Field Names Between Indexes).

  2. Converts Sitecore field values to a format compatible with the Coveo index.

  3. Adds its own set of fields to the Coveo index, which are only used internally. These fields don’t exist in Sitecore.

Coveo fields are grouped within Field Sets. A Field Set defines which fields are indexed in a Coveo source and how they should be handled by the Coveo index. For example, you may want to index your own Sitecore field - for example, productname - and make it free-text searchable, so that any query matching the name of a product would return the said product among search results. To avoid any field name collisions, Coveo for Sitecore creates one Field Set per Coveo source, or if you prefer, one Field Set per Sitecore database that’s indexed.

Coveo fields are most commonly divided into custom and system fields. Coveo already defines several system fields, which for the most part are filled automatically by Coveo: systitle, sysauthor, sysuri, etc. However, custom fields aren’t handled in the same way. As soon as you rebuild your search indexes, Coveo for Sitecore gets all the Sitecore fields that you have added to the Included Fields list and, for each of these fields, one custom Coveo field is created. Coveo for Sitecore names custom fields using a special convention (see Browsing Through Indexed Content and Fields).

Understanding How Fields Work Behind the Scenes

  1. Sitecore fields are mapped to equivalent Coveo fields in the Coveo index.

  2. There’s one Coveo source created for each Sitecore instance database that’s indexed (see Sources section in the Coveo Cloud Console).

  3. Only one Coveo field is created in your Coveo organization when a Sitecore field with given name and settings appears in many sources. When a Sitecore field with given name and settings appears in many sources, the Coveo Cloud Console indicates the number of sources that contain the related Coveo field in the Used in Sources column (see Fields section in the Coveo Cloud Console).

Whenever you change the field settings in the fieldMap section of the Coveo Search Provider configuration file, your field definitions on the Coveo Platform and the mapping assigned to your sources update. Coveo fields are the mechanism that control how Sitecore fields are indexed.

All of your fields can be viewed in the field section of the Coveo Cloud Console, along with their associated settings (see Manage fields).

Understanding How Fields Work Behind the Scenes

  1. Sitecore fields are mapped to equivalent fields in the Coveo index.

  2. Those fields are grouped into Field Sets.

  3. There’s one Field Set assigned to each source in the Coveo index.

  4. There’s one source created for each Sitecore database.

Whenever you change the field settings in the fieldMap section of the Coveo Search Provider configuration file, your Sitecore fields end up in the CES Administration Console, in Configuration > Fields > Custom Fields, along with their associated settings:

Field Set for Coveo_Master_Index

Although you might be tempted to directly modify the settings of the Field Set, doing so is generally discouraged. Instead, use the fieldMap section of the configuration file.

Handling Duplicate Field Names Between Indexes

If a Sitecore field name isn’t configured to be translated, Coveo indexes a single field for both the master and web index, assuming that they have the same settings. However, if you need to separate the fields between indexes, you can configure the field name to be translated.

Field Name Translation

Translating a field name adds an f at the beginning of each field name and a unique index hash at the end of the field name (see Understanding the Coveo Search Provider Configuration File - fieldNames).

For example, when field name productname is not configured to be translated, the field productname is added only once in Coveo Cloud and is shared by the master and web indexes, as well as by any additional manually created index. When field name productname is configured to be translated, a fproductnameXXXXX field is created for each index, where the XXXXX hash is unique to each Sitecore index.

Translating a field name creates the same field for each index, which increases the total amount of indexed fields. Keep your field limit in mind.

Field Name Translation Default Behavior

In pre-September 2018 versions of Coveo for Sitecore 4.1, field name translation is enabled by default. To disable the translation of a field name, you need to explicitly set the field name to isExternal = true inside the fieldMap/fieldNames element of the Coveo.SearchProvider.Custom.config file.

In the September 2018 release of Coveo for Sitecore 4.1 and going forward, you can reverse the default translation behavior by adding the line below in the <settings> element of your Coveo.SearchProvider.config file:

<setting name="Coveo.Indexing.PreferSourceSpecificFields" value="false" />

When this new setting is added, you now need to explicitly specify the field names that you want to translate. To do so, set the field name to isSourceSpecific = true inside the fieldMap/fieldNames element of the Coveo.SearchProvider.Custom.config file.

For more details about the isSourceSpecific and isExternal field attributes and their behavior, see Understanding The Coveo Search Provider Configuration File.

Handling Duplicate Field Names in the Same Index

In Sitecore, nothing prevents you from creating many fields with the same name. If these fields are all of the same type (for example, Single-Line Text), then a single field is created in your search indexes. If an item includes several of these fields and they’re all of type String (for example, Single-Line Text), the item is indexed with a semicolon-separated list of all the values contained in those fields.

However, if you’re indexing two Sitecore fields with the same name but different types (for example, one being of type Single-Line Text, and the other being a Number), then items that include one of these fields might be indexed without the field. When this occurs, you typically see a message such as the following in the Sitecore logs:

WARN  Detected a type change for the field "<SITECORE_FIELDNAME>", but other mappings have been detected. Its type won't be changed and the field won't be usable for this source "<SOURCE_NAME>".

In Sitecore, use a field naming convention which prevents collisions with Coveo system fields from occurring.

Name Collisions With Coveo System Fields

Coveo defines several system fields which, for the most part, are filled automatically by Coveo (for example, title, source, author, uri). Coveo for Sitecore protects these Coveo system field names as well as their values. If you create Sitecore fields whose names collide with Coveo system fields, you might incur issues.

Field names are case insensitive. If you create a field called Author in Sitecore, this name is considered identical to the Coveo author system field.

Fields with names identical to Coveo system fields can be configured to be translated, as any other field (see Field Name Translation). What happens when you have a Sitecore field whose name is the same as a Coveo system field, when the field name isn’t translated and when it’s?

When the Field Name Isn’t Configured to Be Translated

With field name translation disabled, if a Sitecore item includes a field whose name is the same as a Coveo system field, your Coveo Cloud master and web indexed items include that field but the field value is likely to be the one automatically given to the Coveo system field of the same name.

You’ve created a parts template in Sitecore which includes a source field. On an item, you set the value of your source field to ACME. With the source field name not configured to be translated, the indexed item in your Coveo Cloud web source shows the source field. However, its value isn’t ACME but rather the name of your Coveo Cloud web index.

In Sitecore, use a field naming convention which prevents collisions with Coveo system fields from occurring.

When the Field Name Is Configured to Be Translated

With field name translation enabled, if a Sitecore item includes a field whose name is the same as a Coveo system field, your Coveo Cloud master and web indexed items include your translated Sitecore field and the non-translated Coveo system field.

You’ve created a parts template in Sitecore which includes a source field. With the source field name set to isSourceSpecific="true" in Coveo.SearchProvider.config, an indexed item in your Coveo Cloud web source shows both the Coveo source system field (containing the Coveo index name) and your Sitecore field, translated as fsource25249, with the value you set on your Sitecore item.

However, despite Coveo for Sitecore efforts to allow you to create fields in Sitecore with names identical to Coveo system fields, you might notice instances where your translated Sitecore field value gets overwritten by the one in the corresponding Coveo system field. This can occur with the following fields:

  • body

  • clickableUri

  • date

  • documentId

  • fileName

  • fileType

  • id

  • indexedDate

  • modifiedDate

  • orderingId

  • originalSize

  • parentId

  • printableUri

  • retrievedDate

  • size

  • title

In Sitecore, use a field naming convention which prevents collisions with Coveo system fields from occurring.

Understanding When Modified Sitecore Fields Are Updated in the Coveo Index

A Coveo field defines how a Sitecore field is indexed in a Coveo source. Given that Sitecore fields are defined in templates, and that templates can be modified over time (for example, a field is added or removed, or its type is changed), you may wonder exactly when a Coveo field is updated to reflect its modifications. This happens as soon as you re-index any Sitecore item of the database containing the modified template. The end result is that Coveo now has your newly modified field. If it happens that the item that you re-indexed inherits from the template that you modified, it will be indexed along with your newly modified field. However, as a general rule, it’s best to perform a rebuild of your search indexes. This way, you ensure that all items inheriting from the modified template now have your newly modified field.

Controlling Which Fields to Index

Coveo indexes have a field limit, which usually allows much more than what’s needed for a search solution to be relevant.

Hence we recommend that you control the fields to be indexed (see Specifying which Fields to Index).

Changing the Setting of a Coveo Field

When a Sitecore field or computed field is indexed by Coveo, it can be used for field queries or to display its value in an interface. To be able to use the field in more complex controls, such as facets, you must explicitly change the setting of the field (see Changing the Settings of Sitecore Fields).