Field Is No Longer Indexed in the Sitecore Items
Field Is No Longer Indexed in the Sitecore Items
Coveo for Sitecore 4.1 (October 2018)
Symptoms
There are many possible symptoms. Here are just a few:
- A field that had been included in a result template is no longer displayed.
- A facet based on a field is no longer rendered.
- A reference to a field is throwing an exception.
Cause
The October 2018 release of Coveo for Sitecore removed some metadata to reduce the payload of documents sent to the Coveo Cloud.
The table below shows the complete list of legacy fields that were removed.
alt appearancedisplayname appearanceicon appearancelongdescription appearanceribbon appearanceskin appearancesortorder appearancestyle appearancethumbnail branchid browseonly contentpath created createdby databasename datasource datauri description displayname |
empty extension filebased filepath forcemodified fullpath haschildren hasclones hidden hideversion id invalid isclone iscontentitem isexternal isfullyqualified isitemclone ismasterpart ismediaitem |
isvirtual key language layoutcontrol layoutdisplayname layoutfilepath layoutname link longid mediapath mimetype modified name neverpublish originatorid ownerdatabase parentpath path publishdate |
readonly size templatedatabase templateid templatename temporary text title tooltip unpublishdate updated updatedby uri urlstring validfrom validto version workflowstate |
Resolution
Two solutions are available:
- Referencing an alternate field instead
- Adding the legacy metadata into the documents programmatically
Referencing Alternate Fields
You can update your references to the legacy fields to reference alternate fields instead. Just to name a few, those references might be in the following locations:
- Result templates (see
.cshtml
files in<SITECORE_INSTANCE_ROOT>/Website/Coveo/Hive/templates
for Sitecore 7 and 8 instances, or in<SITECORE_INSTANCE_ROOT>/Website/Coveo/Hive/templates
for Sitecore 9 instances) - Computed fields (validate the code in your solution)
Here is a table of removed legacy fields which you have an alternative for:
Removed Legacy Fields | Alternate Field | Differences |
---|---|---|
created | _created | |
createdby | _creator | |
databasename | _database | |
datasource | _datasource | |
editor | _editor | |
fullpath | _fullpath | |
editor | _editor | |
group | _group | |
id | _id | Sample id field value: 33604fcc-da96-4177-a164-e957cea5f928 => Sample _id field value: 33604fccda964177a164e957cea5f928
|
isclone | _isclone | |
language | _language | |
name | _name | |
templateid | _template | Sample templateid field value: f28528eb-0b72-4d70-b476-ffd9f1a169f0 => Sample _template field value: f28528eb0b724d70b476ffd9f1a169f0
|
templatename | _templatename | |
updated | _updated |
Adding the Legacy Metadata into the Documents Programmatically
For your field to be indexed, it needs:
- to be defined in the document metadata, that is, with a processor or with a computed field (see Creating Computed Fields)
- to be included in the
coveoIndexingGetFields
pipeline or defined in your field configuration
You can create a processor in the coveoPostItemProcessingPipeline
that adds legacy metadata back into your documents.
To create a processor that adds the majority of the legacy fields and their values into the documents
- Create a new C# file called
AddLegacyFields.cs
. -
Paste the code below in the
AddLegacyFields.cs
file.using System; using Sitecore.ContentSearch.ComputedFields; using Sitecore.ContentSearch; using Sitecore; using Sitecore.Data.Items; using Sitecore.Data.Fields; using Sitecore.Data; using Coveo.Framework.CNL; using Coveo.Framework.Log; using Coveo.Framework.Pipelines; using Coveo.Framework.Processor; using Coveo.SearchProvider.Pipelines; using Coveo.AbstractLayer.Security; using Coveo.AbstractLayer.RepositoryItem; using Coveo.AbstractLayer.FieldManagement; using System.Linq; using Coveo.Framework.Collections; using System.Collections.Generic; namespace Coveo.AddLegacyFields { public class AddLegacyFieldsMetadata : IProcessor<CoveoPostItemProcessingPipelineArgs> { private static readonly ILogger s_Logger = CoveoLogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType); public bool AddBaseItemFields { get; set; } = true; public bool AddLayoutFields { get; set; } = true; public bool AddMediaItemFields { get; set; } = true; public bool AddItemUriFields { get; set; } = true; public bool AddPathsFields { get; set; } = true; public bool AddStatisticsFields { get; set; } = true; public void Process(CoveoPostItemProcessingPipelineArgs p_Args) { SitecoreIndexableItem sii = p_Args.Item as SitecoreIndexableItem; if (sii != null) { p_Args.OutputCoveoItems.Select(item => { s_Logger.Debug("Adding legacy fields for item ID: " + item.UniqueId.ToString()); return new LegacyMetadataFiller(item); }).ForEach(filler => { if (AddBaseItemFields) filler.AddBaseItemMetadata(sii); if (AddLayoutFields) filler.AddLayoutMetadata(sii); if (AddMediaItemFields) filler.AddMediaItemMetadata(sii); if (AddItemUriFields) filler.AddItemUriMetadata(sii); if (AddPathsFields) filler.AddPathsMetadata(sii); if (AddStatisticsFields) filler.AddStatisticsMetadata(sii); }); } } } internal class LegacyMetadataFiller { private readonly CoveoIndexableItem m_CoveoItem; public LegacyMetadataFiller(CoveoIndexableItem p_IndexableItem) { m_CoveoItem = p_IndexableItem; } public void AddBaseItemMetadata(Item p_Item) { FillWithLegacyItemMetadata(p_Item); } private void FillWithLegacyItemMetadata(Item p_Item) { m_CoveoItem.SetMetadata("BranchId", p_Item.BranchId.ToString()); m_CoveoItem.SetMetadata("DatabaseName", p_Item.Database?.Name); m_CoveoItem.SetMetadata("DisplayName", p_Item.DisplayName); m_CoveoItem.SetMetadata("HasClones", p_Item.HasClones ? "1" : "0"); m_CoveoItem.SetMetadata("IsClone", p_Item.IsClone ? "1" : "0"); m_CoveoItem.SetMetadata("IsItemClone", p_Item.IsItemClone ? "1" : "0"); m_CoveoItem.SetMetadata("Key", p_Item.Key); m_CoveoItem.SetMetadata("Empty", p_Item.Empty ? "1" : "0"); m_CoveoItem.SetMetadata("HasChildren", p_Item.HasChildren ? "1" : "0"); m_CoveoItem.SetMetadata("ID", p_Item.ID.Guid.ToString()); m_CoveoItem.SetMetadata("Modified", p_Item.Modified ? "1" : "0"); m_CoveoItem.SetMetadata("Name", p_Item.Name); m_CoveoItem.SetMetadata("OriginatorID", p_Item.OriginatorId.ToString()); m_CoveoItem.SetMetadata("TemplateID", p_Item.TemplateID.Guid.ToString()); m_CoveoItem.SetMetadata("TemplateName", p_Item.TemplateName); if (p_Item.Language != null) m_CoveoItem.SetMetadata("Language", p_Item.Language.ToString()); if (p_Item.Version != null) m_CoveoItem.SetMetadata("Version", p_Item.Version.ToString()); } public void AddLayoutMetadata(Item p_Item) { LayoutItem layout = p_Item?.Visualization?.Layout; if (layout != null) FillWithLegacyLayoutMetadata(layout); } private void FillWithLegacyLayoutMetadata(LayoutItem p_Layout) { m_CoveoItem.SetMetadata("LayoutControl", p_Layout.Control); m_CoveoItem.SetMetadata("LayoutDisplayName", p_Layout.DisplayName); m_CoveoItem.SetMetadata("LayoutFilePath", p_Layout.FilePath); m_CoveoItem.SetMetadata("LayoutControl", p_Layout.Control); m_CoveoItem.SetMetadata("LayoutName", p_Layout.Name); } public void AddMediaItemMetadata(Item p_Item) { if (p_Item.Paths.IsMediaItem) { MediaItem mediaItem = p_Item; if (mediaItem != null) FillWithLegacyMediaItemMetadata(mediaItem); } } private void FillWithLegacyMediaItemMetadata(MediaItem p_Media) { m_CoveoItem.SetMetadata("Alt", p_Media.Alt); m_CoveoItem.SetMetadata("Extension", p_Media.Extension); m_CoveoItem.SetMetadata("FileBased", p_Media.FileBased); m_CoveoItem.SetMetadata("FilePath", p_Media.FilePath); m_CoveoItem.SetMetadata("Description", p_Media.Description); m_CoveoItem.SetMetadata("Size", p_Media.Size); m_CoveoItem.SetMetadata("Title", p_Media.Title); foreach (KeyValuePair<string, string> metadata in p_Media.GetMetaData()) { if (!String.IsNullOrEmpty(metadata.Key)) { m_CoveoItem.SetMetadata(metadata.Key, metadata.Value); } } } public void AddItemUriMetadata(Item p_Item) { ItemUri uri = p_Item.Uri; if (uri != null) FillWithLegacyItemUriMetadata(uri); } private void FillWithLegacyItemUriMetadata(ItemUri p_Uri) { m_CoveoItem.SetMetadata("DataUri", p_Uri.ToDataUri()?.ToString()); m_CoveoItem.SetMetadata("DefaultUri", p_Uri.ToString()); m_CoveoItem.SetMetadata("UrlString", p_Uri.ToUrlString("")?.ToString()); } public void AddPathsMetadata(Item p_Item) { ItemPath path = p_Item.Paths; if (path != null) FillWithLegacyPathsMetadata(path); } private void FillWithLegacyPathsMetadata(ItemPath p_Path) { m_CoveoItem.SetMetadata("ContentPath", p_Path.ContentPath); m_CoveoItem.SetMetadata("FullPath", p_Path.FullPath); m_CoveoItem.SetMetadata("IsContentItem", p_Path.IsContentItem ? "1" : "0"); m_CoveoItem.SetMetadata("IsFullyQualified", p_Path.IsFullyQualified ? "1" : "0"); m_CoveoItem.SetMetadata("IsMasterPart", p_Path.IsMasterPart ? "1" : "0"); m_CoveoItem.SetMetadata("IsMediaItem", p_Path.IsMediaItem ? "1" : "0"); m_CoveoItem.SetMetadata("LongID", p_Path.LongID); m_CoveoItem.SetMetadata("MediaPath", p_Path.MediaPath); m_CoveoItem.SetMetadata("ParentPath", p_Path.ParentPath); } public void AddStatisticsMetadata(Item p_Item) { ItemStatistics stats = p_Item.Statistics; if (stats != null) FillWithLegacyStatisticsMetadata(stats); } private void FillWithLegacyStatisticsMetadata(ItemStatistics p_Stats) { m_CoveoItem.SetMetadata("Created", p_Stats.Created); m_CoveoItem.SetMetadata("CreatedBy", p_Stats.CreatedBy); m_CoveoItem.SetMetadata("Revision", p_Stats.Revision); m_CoveoItem.SetMetadata("Updated", p_Stats.Updated); m_CoveoItem.SetMetadata("UpdatedBy", p_Stats.UpdatedBy); } } }
- Save and compile the
AddLegacyFields.cs
file in your project.
To create and add a configuration to invoke the processor
- Create a new file using a text editor.
-
Paste the content below in your new file.
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/"> <sitecore> <pipelines> <coveoPostItemProcessingPipeline> <!-- Ensure that the processor type attribute value below matches your compiled namespace and assembly name. --> <processor type="Coveo.AddLegacyFields.AddLegacyFieldsMetadata, AddLegacyFields" /> </coveoPostItemProcessingPipeline> <coveoIndexingGetFields> <processor patch:before="processor[@type='Coveo.AbstractLayer.Processors.Indexing.Fields.IncludeFieldsFromConfigOnlyProcessor, Coveo.AbstractLayer']" type="Coveo.AbstractLayer.Processors.Indexing.Fields.AddCoveoFieldsProcessor, Coveo.AbstractLayer" /> </coveoIndexingGetFields> </pipelines> </sitecore> </configuration>
- Ensure that your processor
type
attribute value matches the compiled namespace and assembly name. - Save the file as
Coveo.z.ComputeLegacyFields.config
in the<SITECORE_INSTANCE_ROOT>\Website\App_Config\Include\Coveo
folder (for Sitecore 7 and 8 instances) or the<SITECORE_INSTANCE_ROOT>\App_Config\Include\Coveo
folder (for Sitecore 9 instances). - Rebuild your indexes.