THIS IS ARCHIVED DOCUMENTATION

Field Is No Longer Indexed in the Sitecore Items

Coveo for Sitecore 4.1 (October 2018)

Symptoms

There are many possible symptoms. Here are just a few:

  • A field that had been included in a result template is no longer displayed.
  • A facet based on a field is no longer rendered.
  • A reference to a field is throwing an exception.

Cause

The October 2018 release of Coveo for Sitecore removed some metadata to reduce the payload of documents sent to the Coveo Cloud.

The table below shows the complete list of legacy fields that were removed.

alt
appearancedisplayname
appearanceicon
appearancelongdescription
appearanceribbon
appearanceskin
appearancesortorder
appearancestyle
appearancethumbnail
branchid
browseonly
contentpath
created
createdby
databasename
datasource
datauri
description
displayname
empty
extension
filebased
filepath
forcemodified
fullpath
haschildren
hasclones
hidden
hideversion
id
invalid
isclone
iscontentitem
isexternal
isfullyqualified
isitemclone
ismasterpart
ismediaitem
isvirtual
key
language
layoutcontrol
layoutdisplayname
layoutfilepath
layoutname
link
longid
mediapath
mimetype
modified
name
neverpublish
originatorid
ownerdatabase
parentpath
path
publishdate
readonly
size
templatedatabase
templateid
templatename
temporary
text
title
tooltip
unpublishdate
updated
updatedby
uri
urlstring
validfrom
validto
version
workflowstate

Resolution

Two solutions are available:

  1. Referencing an alternate field instead
  2. Adding the legacy metadata into the documents programmatically

Referencing Alternate Fields

You can update your references to the legacy fields to reference alternate fields instead. Just to name a few, those references might be in the following locations:

  • Result templates (see .cshtml files in <SITECORE_INSTANCE_ROOT>/Website/Coveo/Hive/templates for Sitecore 7 and 8 instances, or in <SITECORE_INSTANCE_ROOT>/Website/Coveo/Hive/templates for Sitecore 9 instances)
  • Computed fields (validate the code in your solution)

Here is a table of removed legacy fields which you have an alternative for:

Removed Legacy Fields Alternate Field Differences
created _created  
createdby _creator  
databasename _database  
datasource _datasource  
editor _editor  
fullpath _fullpath  
editor _editor  
group _group  
id _id Sample id field value: 33604fcc-da96-4177-a164-e957cea5f928 => Sample _id field value: 33604fccda964177a164e957cea5f928
isclone _isclone  
language _language  
name _name  
templateid _template Sample templateid field value: f28528eb-0b72-4d70-b476-ffd9f1a169f0 => Sample _template field value: f28528eb0b724d70b476ffd9f1a169f0
templatename _templatename  
updated _updated  

Adding the Legacy Metadata into the Documents Programmatically

For your field to be indexed, it needs:

  1. to be defined in the document metadata, that is, with a processor or with a computed field (see Creating Computed Fields)
  2. to be included in the coveoIndexingGetFields pipeline or defined in your field configuration

You can create a processor in the coveoPostItemProcessingPipeline that adds legacy metadata back into your documents.

To create a processor that adds the majority of the legacy fields and their values into the documents

  1. Create a new C# file called AddLegacyFields.cs.
  2. Paste the code below in the AddLegacyFields.cs file.

    using System;
    using Sitecore.ContentSearch.ComputedFields;
    using Sitecore.ContentSearch;
    using Sitecore;
    using Sitecore.Data.Items;
    using Sitecore.Data.Fields;
    using Sitecore.Data;
    using Coveo.Framework.CNL;
    using Coveo.Framework.Log;
    using Coveo.Framework.Pipelines;
    using Coveo.Framework.Processor;
    using Coveo.SearchProvider.Pipelines;
    using Coveo.AbstractLayer.Security;
    using Coveo.AbstractLayer.RepositoryItem;
    using Coveo.AbstractLayer.FieldManagement;
    using System.Linq;
    using Coveo.Framework.Collections;
    using System.Collections.Generic;
    
    namespace Coveo.AddLegacyFields
    {
       
        public class AddLegacyFieldsMetadata : IProcessor<CoveoPostItemProcessingPipelineArgs> 
        {
            private static readonly ILogger s_Logger = CoveoLogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
       
            public bool AddBaseItemFields { get; set; } = true;
            public bool AddLayoutFields { get; set; } = true;
            public bool AddMediaItemFields { get; set; } = true;
            public bool AddItemUriFields { get; set; } = true;
            public bool AddPathsFields { get; set; } = true;
            public bool AddStatisticsFields { get; set; } = true;
       
            public void Process(CoveoPostItemProcessingPipelineArgs p_Args)
            {
                SitecoreIndexableItem sii = p_Args.Item as SitecoreIndexableItem;
                if (sii != null) {
                    p_Args.OutputCoveoItems.Select(item => {
                        s_Logger.Debug("Adding legacy fields for item ID: " + item.UniqueId.ToString());
                        return new LegacyMetadataFiller(item);
                    }).ForEach(filler => {
                        if (AddBaseItemFields) filler.AddBaseItemMetadata(sii);
                        if (AddLayoutFields) filler.AddLayoutMetadata(sii);
                        if (AddMediaItemFields) filler.AddMediaItemMetadata(sii);
                        if (AddItemUriFields) filler.AddItemUriMetadata(sii);
                        if (AddPathsFields) filler.AddPathsMetadata(sii);
                        if (AddStatisticsFields) filler.AddStatisticsMetadata(sii);
                    });
                }
            }
        }
       
        internal class LegacyMetadataFiller 
        {
            private readonly CoveoIndexableItem m_CoveoItem;
            public LegacyMetadataFiller(CoveoIndexableItem p_IndexableItem) 
            {
                m_CoveoItem = p_IndexableItem;
            }
       
            public void AddBaseItemMetadata(Item p_Item) 
            {
                FillWithLegacyItemMetadata(p_Item);
            }
       
            private void FillWithLegacyItemMetadata(Item p_Item) 
            {
                m_CoveoItem.SetMetadata("BranchId", p_Item.BranchId.ToString());
                m_CoveoItem.SetMetadata("DatabaseName", p_Item.Database?.Name);
                m_CoveoItem.SetMetadata("DisplayName", p_Item.DisplayName);
                m_CoveoItem.SetMetadata("HasClones", p_Item.HasClones ? "1" : "0");
                m_CoveoItem.SetMetadata("IsClone", p_Item.IsClone ? "1" : "0");
                m_CoveoItem.SetMetadata("IsItemClone", p_Item.IsItemClone ? "1" : "0");
                m_CoveoItem.SetMetadata("Key", p_Item.Key);
                m_CoveoItem.SetMetadata("Empty", p_Item.Empty ? "1" : "0");
                m_CoveoItem.SetMetadata("HasChildren", p_Item.HasChildren ? "1" : "0");
                m_CoveoItem.SetMetadata("ID", p_Item.ID.Guid.ToString());
                m_CoveoItem.SetMetadata("Modified", p_Item.Modified ? "1" : "0");
                m_CoveoItem.SetMetadata("Name", p_Item.Name);
                m_CoveoItem.SetMetadata("OriginatorID", p_Item.OriginatorId.ToString());
                m_CoveoItem.SetMetadata("TemplateID", p_Item.TemplateID.Guid.ToString());
                m_CoveoItem.SetMetadata("TemplateName", p_Item.TemplateName);
       
                if (p_Item.Language != null) m_CoveoItem.SetMetadata("Language", p_Item.Language.ToString());
                if (p_Item.Version != null) m_CoveoItem.SetMetadata("Version", p_Item.Version.ToString());
            }
       
            public void AddLayoutMetadata(Item p_Item) 
            {
                LayoutItem layout = p_Item?.Visualization?.Layout;
                if (layout != null) FillWithLegacyLayoutMetadata(layout);
            }
       
            private void FillWithLegacyLayoutMetadata(LayoutItem p_Layout) 
            {
                m_CoveoItem.SetMetadata("LayoutControl", p_Layout.Control);
                m_CoveoItem.SetMetadata("LayoutDisplayName", p_Layout.DisplayName);
                m_CoveoItem.SetMetadata("LayoutFilePath", p_Layout.FilePath);
                m_CoveoItem.SetMetadata("LayoutControl", p_Layout.Control);
                m_CoveoItem.SetMetadata("LayoutName", p_Layout.Name);
            }
       
            public void AddMediaItemMetadata(Item p_Item) 
            {
                if (p_Item.Paths.IsMediaItem) {
                    MediaItem mediaItem = p_Item;
                    if (mediaItem != null) FillWithLegacyMediaItemMetadata(mediaItem);
                }
            }
       
            private void FillWithLegacyMediaItemMetadata(MediaItem p_Media) 
            {
                m_CoveoItem.SetMetadata("Alt", p_Media.Alt);
                m_CoveoItem.SetMetadata("Extension", p_Media.Extension);
                m_CoveoItem.SetMetadata("FileBased", p_Media.FileBased);
                m_CoveoItem.SetMetadata("FilePath", p_Media.FilePath);
                m_CoveoItem.SetMetadata("Description", p_Media.Description);
                m_CoveoItem.SetMetadata("Size", p_Media.Size);
                m_CoveoItem.SetMetadata("Title", p_Media.Title);
       
                foreach (KeyValuePair<string, string> metadata in p_Media.GetMetaData()) 
                {
                    if (!String.IsNullOrEmpty(metadata.Key)) {
                        m_CoveoItem.SetMetadata(metadata.Key, metadata.Value);
                    }
                }
            }
       
            public void AddItemUriMetadata(Item p_Item) 
            {
                ItemUri uri = p_Item.Uri;
                if (uri != null) FillWithLegacyItemUriMetadata(uri);
            }
       
            private void FillWithLegacyItemUriMetadata(ItemUri p_Uri) 
            {
                m_CoveoItem.SetMetadata("DataUri", p_Uri.ToDataUri()?.ToString());
                m_CoveoItem.SetMetadata("DefaultUri", p_Uri.ToString());
                m_CoveoItem.SetMetadata("UrlString", p_Uri.ToUrlString("")?.ToString());
            }
       
            public void AddPathsMetadata(Item p_Item) {
                ItemPath path = p_Item.Paths;
                if (path != null) FillWithLegacyPathsMetadata(path);
            }
       
            private void FillWithLegacyPathsMetadata(ItemPath p_Path) 
            {
                m_CoveoItem.SetMetadata("ContentPath", p_Path.ContentPath);
                m_CoveoItem.SetMetadata("FullPath", p_Path.FullPath);
                m_CoveoItem.SetMetadata("IsContentItem", p_Path.IsContentItem ? "1" : "0");
                m_CoveoItem.SetMetadata("IsFullyQualified", p_Path.IsFullyQualified ? "1" : "0");
                m_CoveoItem.SetMetadata("IsMasterPart", p_Path.IsMasterPart ? "1" : "0");
                m_CoveoItem.SetMetadata("IsMediaItem", p_Path.IsMediaItem ? "1" : "0");
                m_CoveoItem.SetMetadata("LongID", p_Path.LongID);
                m_CoveoItem.SetMetadata("MediaPath", p_Path.MediaPath);
                m_CoveoItem.SetMetadata("ParentPath", p_Path.ParentPath);
            }
       
            public void AddStatisticsMetadata(Item p_Item) 
            {
                ItemStatistics stats = p_Item.Statistics;
                if (stats != null) FillWithLegacyStatisticsMetadata(stats);
            }
       
            private void FillWithLegacyStatisticsMetadata(ItemStatistics p_Stats) 
            {
                m_CoveoItem.SetMetadata("Created", p_Stats.Created);
                m_CoveoItem.SetMetadata("CreatedBy", p_Stats.CreatedBy);
                m_CoveoItem.SetMetadata("Revision", p_Stats.Revision);
                m_CoveoItem.SetMetadata("Updated", p_Stats.Updated);
                m_CoveoItem.SetMetadata("UpdatedBy", p_Stats.UpdatedBy);
            }
        }
    }
    
  3. Save and compile the AddLegacyFields.cs file in your project.

To create and add a configuration to invoke the processor

  1. Create a new file using a text editor.
  2. Paste the content below in your new file.

    <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
      <sitecore>
        <pipelines>
          <coveoPostItemProcessingPipeline>
            <!-- Ensure that the processor type attribute value below matches your compiled namespace and assembly name. -->
            <processor type="Coveo.AddLegacyFields.AddLegacyFieldsMetadata, AddLegacyFields" />
          </coveoPostItemProcessingPipeline>
          <coveoIndexingGetFields>
            <processor
              patch:before="processor[@type='Coveo.AbstractLayer.Processors.Indexing.Fields.IncludeFieldsFromConfigOnlyProcessor, Coveo.AbstractLayer']"
              type="Coveo.AbstractLayer.Processors.Indexing.Fields.AddCoveoFieldsProcessor, Coveo.AbstractLayer" />
          </coveoIndexingGetFields>
        </pipelines>
      </sitecore>
    </configuration>
    
  3. Ensure that your processor type attribute value matches the compiled namespace and assembly name.
  4. Save the file as Coveo.z.ComputeLegacyFields.config in the <SITECORE_INSTANCE_ROOT>\Website\App_Config\Include\Coveo folder (for Sitecore 7 and 8 instances) or the <SITECORE_INSTANCE_ROOT>\App_Config\Include\Coveo folder (for Sitecore 9 instances).
  5. Rebuild your indexes.