Field Is No Longer Indexed in the Sitecore Items

Coveo for Sitecore 4.1 (October 2018)

Symptoms

There are many possible symptoms. Here are just a few:

• A field that had been included in a result template is no longer displayed
• A facet based on a field is no longer rendered
• A reference to a field is throwing an exception

Cause

The October 2018 release of Coveo for Sitecore removed some metadata to reduce the payload of documents sent to the Coveo Cloud.

Resolution

Two solutions are available:

1. Reference alternative fields instead.
2. Adding the legacy metadata into the documents programmatically.

Reference Alternative Fields

You can update your references to the legacy fields to reference alternative fields instead. Just to name a few, those references might be located in:

• Result templates (see .cshtml files in [Your_Sitecore_Instance]Website/Coveo/Hive/templates for Sitecore 7 and 8 instances, or in [Your_Sitecore_Instance]Website/Coveo/Hive/templates for Sitecore 9 instances).
• Computed fields (validate the code in your solution)

Here is a table of the legacy fields and their alternatives.

Here is the list of removed fields that must instead be added using a computed field:

• branchid
• datauri
• displayname
• empty
• hasclones
• isitemclone
• key
• layoutdisplayname
• layoutfilepath
• layoutname
• originatorid
• revision
• updatedby
• urlstring
• workflowstate

Adding the Legacy Metadata into the Documents Programmatically

You can create a processor in the coveoPostItemProcessingPipeline that adds the legacy metadata back into your documents.

For your field to be indexed, it needs:

1. To be defined in the document metadata. For instance, with a processor, or with computed fields
2. To be defined in your field configuration or included in the coveoIndexingGetFields pipeline.

To create a processor that adds the majority of the legacy fields and their values into the documents

1. Create a new c# file called AddLegacyFields.cs.

2. Paste the code below in the AddLegacyFields.cs file.

   using System;
   namespace Coveo.AddLegacyFields {
       using Sitecore.ContentSearch.ComputedFields;
       using Sitecore.ContentSearch;
       using Sitecore;
       using Sitecore.Data.Items;
       using Sitecore.Data.Fields;
       using Sitecore.Data;
       using Coveo.Framework.CNL;
       using Coveo.Framework.Log;
       using Coveo.Framework.Pipelines;
       using Coveo.Framework.Processor;
       using Coveo.SearchProvider.Pipelines;
       using Coveo.AbstractLayer.Security;
       using Coveo.AbstractLayer.RepositoryItem;
       using Coveo.AbstractLayer.FieldManagement;
       using System.Linq;
       using Coveo.Framework.Collections;
       using System.Collections.Generic;
       public class AddLegacyFieldsMetadata : IProcessor<CoveoPostItemProcessingPipelineArgs> {
           private static readonly ILogger s_Logger = CoveoLogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
           public bool AddBaseItemFields { get; set; } = true;
           public bool AddLayoutFields { get; set; } = true;
           public bool AddMediaItemFields { get; set; } = true;
           public bool AddItemUriFields { get; set; } = true;
           public bool AddPathsFields { get; set; } = true;
           public bool AddStatisticsFields { get; set; } = true;
           public void Process(CoveoPostItemProcessingPipelineArgs p_Args)
           {
               SitecoreIndexableItem sii = p_Args.Item as SitecoreIndexableItem;
               if (sii != null) {
                   p_Args.OutputCoveoItems.Select(item => {
                       s_Logger.Debug("Adding legacy fields for item ID: " + item.UniqueId.ToString());
                       return new LegacyMetadataFiller(item);
                   }).ForEach(filler => {
                       if (AddBaseItemFields) filler.AddBaseItemMetadata(sii);
                       if (AddLayoutFields) filler.AddLayoutMetadata(sii);
                       if (AddMediaItemFields) filler.AddMediaItemMetadata(sii);
                       if (AddItemUriFields) filler.AddItemUriMetadata(sii);
                       if (AddPathsFields) filler.AddPathsMetadata(sii);
                       if (AddStatisticsFields) filler.AddStatisticsMetadata(sii);
                   });
               }
           }
       }
       internal class LegacyMetadataFiller {
           private readonly CoveoIndexableItem m_CoveoItem;
           public LegacyMetadataFiller(CoveoIndexableItem p_IndexableItem) {
               m_CoveoItem = p_IndexableItem;
           }
           public void AddBaseItemMetadata(Item p_Item) {
               FillWithLegacyItemMetadata(p_Item);
           }
           private void FillWithLegacyItemMetadata(Item p_Item) {
               m_CoveoItem.SetMetadata("BranchId", p_Item.BranchId.ToString());
               m_CoveoItem.SetMetadata("DatabaseName", p_Item.Database?.Name);
               m_CoveoItem.SetMetadata("DisplayName", p_Item.DisplayName);
               m_CoveoItem.SetMetadata("HasClones", p_Item.HasClones ? "1" : "0");
               m_CoveoItem.SetMetadata("IsClone", p_Item.IsClone ? "1" : "0");
               m_CoveoItem.SetMetadata("IsItemClone", p_Item.IsItemClone ? "1" : "0");
               m_CoveoItem.SetMetadata("Key", p_Item.Key);
               m_CoveoItem.SetMetadata("Empty", p_Item.Empty ? "1" : "0");
               m_CoveoItem.SetMetadata("HasChildren", p_Item.HasChildren ? "1" : "0");
               m_CoveoItem.SetMetadata("ID", p_Item.ID.Guid.ToString());
               m_CoveoItem.SetMetadata("Modified", p_Item.Modified ? "1" : "0");
               m_CoveoItem.SetMetadata("Name", p_Item.Name);
               m_CoveoItem.SetMetadata("OriginatorID", p_Item.OriginatorId.ToString());
               m_CoveoItem.SetMetadata("TemplateID", p_Item.TemplateID.Guid.ToString());
               m_CoveoItem.SetMetadata("TemplateName", p_Item.TemplateName);
               if (p_Item.Language != null) m_CoveoItem.SetMetadata("Language", p_Item.Language.ToString());
               if (p_Item.Version != null) m_CoveoItem.SetMetadata("Version", p_Item.Version.ToString());
           }
           public void AddLayoutMetadata(Item p_Item) {
               LayoutItem layout = p_Item?.Visualization?.Layout;
               if (layout != null) FillWithLegacyLayoutMetadata(layout);
           }
           private void FillWithLegacyLayoutMetadata(LayoutItem p_Layout) {
               m_CoveoItem.SetMetadata("LayoutControl", p_Layout.Control);
               m_CoveoItem.SetMetadata("LayoutDisplayName", p_Layout.DisplayName);
               m_CoveoItem.SetMetadata("LayoutFilePath", p_Layout.FilePath);
               m_CoveoItem.SetMetadata("LayoutControl", p_Layout.Control);
               m_CoveoItem.SetMetadata("LayoutName", p_Layout.Name);
           }
           public void AddMediaItemMetadata(Item p_Item) {
               if (p_Item.Paths.IsMediaItem) {
                   MediaItem mediaItem = p_Item;
                   if (mediaItem != null) FillWithLegacyMediaItemMetadata(mediaItem);
               }
           }
           private void FillWithLegacyMediaItemMetadata(MediaItem p_Media) {
               m_CoveoItem.SetMetadata("Alt", p_Media.Alt);
               m_CoveoItem.SetMetadata("Extension", p_Media.Extension);
               m_CoveoItem.SetMetadata("FileBased", p_Media.FileBased);
               m_CoveoItem.SetMetadata("FilePath", p_Media.FilePath);
               m_CoveoItem.SetMetadata("Description", p_Media.Description);
               m_CoveoItem.SetMetadata("Size", p_Media.Size);
               m_CoveoItem.SetMetadata("Title", p_Media.Title);
               foreach (KeyValuePair<string, string> metadata in p_Media.GetMetaData()) {
                   if (!String.IsNullOrEmpty(metadata.Key)) {
                       m_CoveoItem.SetMetadata(metadata.Key, metadata.Value);
                   }
               }
           }
           public void AddItemUriMetadata(Item p_Item) {
               ItemUri uri = p_Item.Uri;
               if (uri != null) FillWithLegacyItemUriMetadata(uri);
           }
           private void FillWithLegacyItemUriMetadata(ItemUri p_Uri) {
               m_CoveoItem.SetMetadata("DataUri", p_Uri.ToDataUri()?.ToString());
               m_CoveoItem.SetMetadata("DefaultUri", p_Uri.ToString());
               m_CoveoItem.SetMetadata("UrlString", p_Uri.ToUrlString("")?.ToString());
           }
           public void AddPathsMetadata(Item p_Item) {
               ItemPath path = p_Item.Paths;
               if (path != null) FillWithLegacyPathsMetadata(path);
           }
           private void FillWithLegacyPathsMetadata(ItemPath p_Path) {
               m_CoveoItem.SetMetadata("ContentPath", p_Path.ContentPath);
               m_CoveoItem.SetMetadata("FullPath", p_Path.FullPath);
               m_CoveoItem.SetMetadata("IsContentItem", p_Path.IsContentItem ? "1" : "0");
               m_CoveoItem.SetMetadata("IsFullyQualified", p_Path.IsFullyQualified ? "1" : "0");
               m_CoveoItem.SetMetadata("IsMasterPart", p_Path.IsMasterPart ? "1" : "0");
               m_CoveoItem.SetMetadata("IsMediaItem", p_Path.IsMediaItem ? "1" : "0");
               m_CoveoItem.SetMetadata("LongID", p_Path.LongID);
               m_CoveoItem.SetMetadata("MediaPath", p_Path.MediaPath);
               m_CoveoItem.SetMetadata("ParentPath", p_Path.ParentPath);
           }
           public void AddStatisticsMetadata(Item p_Item) {
               ItemStatistics stats = p_Item.Statistics;
               if (stats != null) FillWithLegacyStatisticsMetadata(stats);
           }
           private void FillWithLegacyStatisticsMetadata(ItemStatistics p_Stats) {
               m_CoveoItem.SetMetadata("Created", p_Stats.Created);
               m_CoveoItem.SetMetadata("CreatedBy", p_Stats.CreatedBy);
               m_CoveoItem.SetMetadata("Revision", p_Stats.Revision);
               m_CoveoItem.SetMetadata("Updated", p_Stats.Updated);
               m_CoveoItem.SetMetadata("UpdatedBy", p_Stats.UpdatedBy);
           }
       }
   }
   

3. Save and compile the AddLegacyFields.cs file in your project.

To create and add a configuration to invoke the processor

1. Create a new file using a text editor.

2. Paste the content below in your new file.

   <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
     <sitecore>
       <pipelines>
         <coveoPostItemProcessingPipeline>
           <!-- Ensure the processor type attribute value below matches your compiled namespace and assembly name. -->
           <processor type="Coveo.AddLegacyFields.AddLegacyFieldsMetadata, AddLegacyFields" />
         </coveoPostItemProcessingPipeline>
         <coveoIndexingGetFields>
           <processor
             patch:before="processor[@type='Coveo.AbstractLayer.Processors.Indexing.Fields.IncludeFieldsFromConfigOnlyProcessor, Coveo.AbstractLayer']"
             type="Coveo.AbstractLayer.Processors.Indexing.Fields.AddCoveoFieldsProcessor, Coveo.AbstractLayer" />
         </coveoIndexingGetFields>
       </pipelines>
     </sitecore>
   </configuration>
   

3. Ensure your processor type attribute value matches the compiled namespace and assembly name.
4. Save the file as Coveo.z.ComputeLegacyFields.config in the [Your_Sitecore_Instance]\Website\App_Config\Include\Coveo folder (for Sitecore 7 and 8 instances) or the [Your_Sitecore_Instance]\App_Config\Include\Coveo folder (for Sitecore 9 instances).
5. Rebuild your indexes.