---
title: Index XML sitemap metadata
slug: '2656'
canonical_url: https://docs.coveo.com/en/2656/
collection: index-content
source_format: adoc
---
# Index XML sitemap metadata
In addition to the various webpage [metadata](https://docs.coveo.com/en/218/) extraction features, the [Sitemap](https://docs.coveo.com/en/1967/) [source](https://docs.coveo.com/en/246/) supports extracting [metadata](https://docs.coveo.com/en/218/) from an XML sitemap file.
This metadata can come from:

* Third-party sitemap extensions, such as the Google image extension, which provide Google and other sitemap crawlers with further information regarding the images your site contains.
See [Extending the Sitemaps protocol](https://www.sitemaps.org/protocol.html#extending), [Image sitemaps](https://support.google.com/webmasters/answer/178636?hl=en&ref_topic=4581190), and [Third-party extensions](#third-party-extensions) for more details.

* A Coveo namespace (`coveo:metadata`) and metadata added to your sitemap to provide Coveo with additional information on [items](https://docs.coveo.com/en/210/) (see [Coveo-specific custom metadata](#coveo-specific-custom-metadata)).

In any case, the [steps to configure how Coveo stores this information](#configuring-fields-and-mappings) are the same.

## Third-party extensions

Some sites such as Google offer extensions adding extra metadata to your sitemap (see [Image sitemaps](https://support.google.com/webmasters/answer/178636?hl=en&ref_topic=4581190)).
Alternatively, you can build your own extension [Extending the Sitemaps protocol](https://www.sitemaps.org/protocol.html#extending).
Either way, the data added to your sitemap can be retrieved and made searchable by Coveo.
See [Configuring fields and mappings](#configuring-fields-and-mappings) to configure Coveo adequately.

See also [Video sitemaps and video sitemap alternatives](https://support.google.com/webmasters/answer/80471?hl=en) for another example.

## Coveo-specific custom metadata

A developer can include custom metadata in an XML sitemap file specifically for Coveo indexing purposes.
When they can generate or modify the sitemap XML file of a repository to index, they can also include a Coveo namespace (`coveo:metadata`) and metadata to provide information on items that isn't found in default fields (that is, Sitemap standard source fields and Coveo default fields).

**Example**

Since you have control on the sitemap file (it's not generated by a third party), you decide to create your XML sitemap file dynamically and add all the custom metadata you need.

Although the added Coveo metadata will only be read by the Coveo crawler and connector and ignored by all other processes, it still respects the Sitemap protocol (see [Sitemaps XML format](https://www.sitemaps.org/protocol.html#extending)).

> **Important**
>
> Google is known to have issues with `<coveo:metadata>` tags in sitemaps.
> Google Search Console may report `Invalid XML tag` errors when `<coveo:metadata>` tags are present.
> To inject custom metadata without affecting SEO, include your `<coveo:metadata>` tags in a separate copy of the sitemap used only for Coveo indexing, one that requires authentication to access.

The following procedure requires a user that has the permissions and skills to modify or create an XML sitemap file and the required privileges in the Coveo Administration Console.

To add Coveo-specific custom metadata in an XML sitemap

You must code a third-party process to modify or create an XML sitemap file as follows:

. In the `urlset` XML element start tag (`<urlset>`), extend the Sitemap protocol using the Coveo namespace by adding the following line:

`+xmlns:coveo="https://www.coveo.com/en/company/about-us"+`

> **Note**
>
> From a Coveo perspective, the value of the `xmlns:coveo` attribute (that is, the URI) is irrelevant.
> The Coveo sitemap crawler ignores this value.
> However, other web search engine indexing services may need to validate this URI.
> 
> The attribute name (that is, `xmlns:coveo`) is important as the sitemap XML file will contain elements in the `coveo` namespace scope.

**Example**

```xml
<?xml version="1.0" encoding="UTF-8"?>
<urlset
  xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
  xmlns:coveo="https://www.coveo.com/en/company/about-us">
```

. For each url element (`<url></url>`) in the sitemap, create a new XML element named coveo:metadata (`<coveo:metadata></coveo:metadata>`).

**Example**

```xml
<url>
  <loc>http://example.com/about/</loc>
  <lastmod>2015-02-10T13:47:23+00:00</lastmod>
  <changefreq>weekly</changefreq>
  <priority>1.00</priority>
  <coveo:metadata>
  </coveo:metadata>
</url>
```

. Within the `coveo:metadata` elements, add your custom metadata (name and value).

> **Notes**
>
> * To index special characters, use a CDATA tag (`![CDATA[`) at the beginning of the node (see [Character Data and Markup](https://www.w3.org/TR/REC-xml/#dt-chardata)).
> The source then ignores the CDATA tag and indexes the rest of the node content such as special characters (for example, `&`, `%`, `$`, `~`, and `<xml>` tags) as text.
>  
> Example:
>  
> The `companyname` metadata in the following sitemap file content
>  
> ```xml
<coveo:metadata>
  <casenumber>18467</casenumber>
  <companyname>
    <![CDATA[
    Company XYZ Inc. <USA>
    ]]>
  </companyname>
</coveo:metadata>
```
>  
> is indexed as follows in your Coveo index:
>  
> ![Indexing sitemap metadata when adding CDATA tag](:https://docs.coveo.com/en/assets/images/index-content/sitemap-metadata-with-c-data-tag-indexing.png)
> 
> * Nested metadata inside the `<coveo:metadata>` element isn't supported.

**Example**

You want to add the name of the author, the last date of modification and the document tags (if any) so you add the following XML elements:

```xml
<coveo:metadata>
  <modificationdate>2015-02-10T13:47:23+00:00</modificationdate>
  <authorname>John Smith</authorname>
  <tags />
</coveo:metadata>
```

Once done, the sitemap could look like the following:

```xml
<?xml version="1.0" encoding="UTF-8"?>
<urlset
  xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"
  xmlns:coveo="https://www.example.com/schemas">
  <url>
    <loc>http://example.com/about/</loc>
    <lastmod>2015-02-10T13:47:23+00:00</lastmod>
    <changefreq>weekly</changefreq>
    <priority>1.00</priority>
    <coveo:metadata>
      <modificationdate>2015-02-10T13:47:23+00:00</modificationdate>
      <authorname>John Smith</authorname>
      <tags />
    </coveo:metadata>
  </url>
</urlset>
```

> **Note**
>
> For more information, contact [Coveo Professional Services](https://www.coveo.com/en/services/professional-services).

## Indexing a Sitemap source by reference

By default, a Sitemap source is set to `retrieve` HTML and PDF items (that is, to index their content and metadata).
With the document content, the [Coveo Platform](https://docs.coveo.com/en/186/) produces the item [quickview](https://docs.coveo.com/en/3311/), [excerpt](https://docs.coveo.com/en/3310/), and summary.

If you don't need the [quickview](https://docs.coveo.com/en/3311/), [excerpt](https://docs.coveo.com/en/3310/), and summary, and you have all the information you want to index in your sitemap file metadata, you may want to only [index the metadata](https://docs.coveo.com/en/l3qg9275/).
This improves performance.

> **Important**
>
> Indexing by reference _doesn't_ mean your [web scraping configuration](https://docs.coveo.com/en/1967#web-scraping-subtab) is ignored.
> The Coveo sitemap crawler will still scrape the content of documents matching your [exclusion and inclusion rules](https://docs.coveo.com/en/1967#exclusions-and-inclusions).
> To prevent unexpected field values, avoid using the same metadata names in your web scraping configuration as the ones in your sitemap file.

To index a Sitemap source by reference

. On the [**Sources**](https://platform.cloud.coveo.com/admin/#/orgid/content/sources/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/content/sources/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/content/sources/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/content/sources/)) page of the [Coveo Administration Console](https://docs.coveo.com/en/183/), [add a Sitemap source](https://docs.coveo.com/en/1967/).

. [Access the **Edit configuration with JSON** panel](https://docs.coveo.com/en/1685#access-the-edit-configuration-with-json-panel) of the source you just created.

. In the `documentConfig` section of the JSON source configuration, find the `extensionSettings` section.

. In the `extensionSettings` section, delete the `ByExtensions` and `ByContentTypes` sections.

![JSON configuration with byExtensions and byContentTypes sections highlighted | Coveo](https://docs.coveo.com/en/assets/images/index-content/by-extension.png)

. Find the `noExtension` and the `other` sections.

--
** In the `noExtension` section, change the `action` value from `Retrieve` to `Reference`.

** In the `other` section, change the `action` value from `Retrieve` to `Reference`.
--

![JSON configuration with Reference actions highlighted](https://docs.coveo.com/en/assets/images/index-content/indexing-by-reference-json-action.png)

. Click **Save and Rebuild Source**.

## Configuring fields and mappings

Regardless of how the additional metadata was added to your sitemap, you must configure Coveo so that it indexes this information adequately.

. In the [Coveo Administration Console](https://docs.coveo.com/en/183/), ensure that you have the [required privileges](https://docs.coveo.com/en/3390#required-privileges).

. On the [**Fields**](https://platform.cloud.coveo.com/admin/#/orgid/content/fields/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/content/fields/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/content/fields/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/content/fields/)) page, for each metadata you want to see in your item details, [add the corresponding custom field](https://docs.coveo.com/en/1833#add-a-field).

. On the [**Sources**](https://platform.cloud.coveo.com/admin/#/orgid/content/sources/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/content/sources/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/content/sources/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/content/sources/)) page, [add a mapping rule](https://docs.coveo.com/en/1640#manage-mappings) for each field you added.

> **Important**
>
> * The **metadataName** in mapping rules must match the XML element name.
> 
> * XML element names are case-sensitive.

> **Notes**
>
> * Coveo supports a single level of metadata in the `<coveo:metadata>` element.
>  
> For example:
>  
> ```xml
<coveo:metadata>
  <OSCodes>WW1</OSCodes>
  <product>Inspiron XPS;Dimension XPS</product>
</coveo:metadata>
```
> 
> * Coveo supports extensions to the Sitemap standard (for example, the [Google Video Sitemap](https://developers.google.com/search/docs/advanced/sitemaps/video-sitemaps)).
> In this scenario, Coveo flattens the metadata, that is, the key of each piece of data is the result of the path to the corresponding value.
>  
> For example, the sitemap excerpt below results in the following flattened metadata: `+"video.thumbnail_loc": "http://img.youtube.com/vi/wejYF7l0kKQ/2.jpg"+`.
>  
> ```xml
  <url>
    <loc>http://www.example.com/videos/some_video_landing_page.html</loc>
    <video:video>
      <video:thumbnail_loc>
        http://img.youtube.com/vi/wejYF7l0kKQ/2.jpg
      </video:thumbnail_loc>
    </video:video>
  </url>
```

**Example**

You want to have the video thumbnail in the results metadata, so you add the `videothumbnail` field and use the following mapping rule: `%[video.thumbnail_loc]`.

![Mapping rule configuration for a Sitemap source in the Coveo Administration Console](https://docs.coveo.com/en/assets/images/index-content/mapping-rule-for-sitemap-source.png)

. Save and rebuild your Sitemap source.

. On the [**Content Browser**](https://platform.cloud.coveo.com/admin/#/orgid/content/browser/) ([platform-ca](https://platform-ca.cloud.coveo.com/admin/#/orgid/content/browser/) | [platform-eu](https://platform-eu.cloud.coveo.com/admin/#/orgid/content/browser/) | [platform-au](https://platform-au.cloud.coveo.com/admin/#/orgid/content/browser/)) page, on the [**Fields** tab](https://docs.coveo.com/en/1712#access-the-fields-tab) located in the **Properties** panel of your Sitemap source items, ensure that the new metadata is available.