Changing the Crawling Root of an Index

Changing the Crawling Root of an Index

By default, Coveo for Sitecore indexes all items under /sitecore/content, as well as all items under /sitecore/media library/Files. You may want to change the crawling root of your indexes, most often to prevent indexing undesirable items in specific indexes. This can be particularly useful in multi-site installations, or if you have indexes dedicated to certain content.

By changing the crawling root, you can focus the indexing on an item and its content tree, which can accelerate your indexing time.

Modify a Crawling Root

To change the crawling root of your indexes, follow these steps:

Change the Crawling Root

1. Open your Coveo.SearchProvider.Custom.config file. It’s located under <Sitecore Instance>\Website\App_config\Include\Coveo.

2. At the end of the sitecore node, before the scheduling node, add the following elements:

    <contentSearch>
      <configuration>
        <indexes>
          <index id="Coveo_web_index">
            <locations>
              <crawler name="ContentCrawler">
                <root>/sitecore/content</root>
              </crawler>
            </locations>
          </index>
        </indexes>
      </configuration>
    </contentSearch>
   

3. Change the index id to the index you want to modify.

4. Change the name of the crawler to match the crawler you want to modify. The default names are ContentCrawler for items under /sitecore/content and MediaItemCrawler for items under /sitecore/media library/Files.

5. Change the root node to the one you want.

   Ensure that the path you enter includes:

   • Your Coveo-powered search page for all Coveo indexes.

   • Any page containing a Coveo UI component (such as a search box or recommendations).

   • Any datasource that’s feeding Coveo UI components.

   Failure to do so results in the following error: Precondition failed: The parameter 'p_Index' must not be null.

   If your desired root node doesn’t include all these pages, you may want to add a new crawling root to specifically include them (see Add a New Crawling Root).

6. Save and close your file.

Name Your Crawlers

Coveo for Sitecore (July 2016)

In releases prior to July 2016, the index crawlers are unnamed, which can complicate their modification. To name your crawlers, follow this procedure:

1. Open your Coveo.SearchProvider.config file. It’s located under <Sitecore Instance>\Website\App_config\Include\Coveo.

2. Under indexes > index > locations, add a name to your crawler. Crawlers with the root /sitecore/content should be named ContentCrawler, while crawlers with the root /sitecore/media library/Files should be named MediaLibraryCrawler.

   By default, there are two indexes (master and web) with two crawlers each. If you added new indexes, make sure that you name their crawlers too.

   You should avoid modifying the Coveo.SearchProvider.config file as it can cause unforeseen upgrading issues. Therefore, it’s important that you respect the crawler name and casing.

   Sitecore 8.0

    <crawler name="ContentCrawler" type="Coveo.SearchProvider.CoveoItemCrawler, Coveo.SearchProviderBase">
   

   Sitecore 8.1

    <crawler name="ContentCrawler" type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
   

3. Save and close the file.

Add a New Crawling Root

You may also want to add a new crawling root to your indexes. To do so, follow these steps:

1. Open your Coveo.SearchProvider.Custom.config file. It’s located under <Sitecore Instance>\Website\App_config\Include\Coveo.

2. In the locations node, add the following element.

   Sitecore 8.0

    <crawler name="CrawlerName" type="Coveo.SearchProvider.CoveoItemCrawler, Coveo.SearchProviderBase">
      <database>web</database>
      <root>/sitecore/content</root>
      <stopOnError>true</stopOnError>
    </crawler>
   

   Sitecore 8.1

    <crawler name="CrawlerName" type="Sitecore.ContentSearch.SitecoreItemCrawler, Sitecore.ContentSearch">
      <database>web</database>
      <root>/sitecore/content</root>
      <stopOnError>true</stopOnError>
    </crawler>
   

3. Change the name of the crawler to one you deem appropriate.

   You should limit the name of your crawler to alphanumeric characters.

4. In the database node, enter the name of the database you want to add your crawler to.

5. Change the root node to your desired crawling root.

6. Save and close your file.

Test Your New Configuration

Now that you have changed or added your index crawling root, you need to verify that it works.

Test your new configuration by re-indexing your modified index (see Coveo for Sitecore Indexing Guide). You should notice a change in the number of indexed items that reflects your modifications.