---
title: Analyze the rebuild process
slug: '2147'
canonical_url: https://docs.coveo.com/en/2147/
collection: coveo-for-sitecore-v5
source_format: adoc
---
# Analyze the rebuild process
Rebuilding is the action of crawling a set of items and pushing them into your [Coveo organization](https://docs.coveo.com/en/185/) [source](https://docs.coveo.com/en/246/).
At the end of the process, the source is expected to contain only the crawled items.
A Coveo index rebuild in Sitecore, whether launched through the Sitecore user interface or the [Coveo Command Center](https://docs.coveo.com/en/2426#rebuilding-a-search-index-manually), includes many steps.
Items are first handled by Sitecore and then handed over to Coveo for Sitecore.
The best way to understand which Sitecore items are indexed, and how they're indexed, is to monitor activities using the Coveo rebuild logs.

This article breaks down Coveo index rebuild log messages and explains what Coveo for Sitecore is actually doing as these messages are being recorded.
This article also shows how you can isolate Coveo rebuild logs in a separate file and change the log level to help you diagnose rebuild issues.

## Coveo rebuild phases

[Coveo rebuild logs](#coveo-rebuild-logs) are divided into four phases, the same phases you see in the rebuild status bar during a [**Command Center** rebuild](https://docs.coveo.com/en/2426#rebuilding-a-search-index-manually).

![Image of indexing phases in the Command Center during a rebuild | Coveo for Sitecore 5](https://docs.coveo.com/en/assets/images/c4sc-v5/command-center-indexing-step-skipped-numbered.png)

### ![Number one icon for Setup Requirements phase | Coveo](c4sc-v5/one.svg) Setup Requirements phase

During this phase, the Sitecore configuration is compared to the resources in the [Coveo Platform](https://docs.coveo.com/en/186/).
The sources, fields, and security providers are modified in Coveo to match Sitecore configurations.

### ![Number two icon representing the Sending Permissions phase | Coveo](c4sc-v5/two.svg) Sending Permissions phase

During this phase, all Sitecore users and roles are sent to Coveo.
This way, Coveo doesn't have to contact the Sitecore instance at query time.

### ![Number three icon representing Sending Documents phase | Coveo](c4sc-v5/three.svg) Sending Documents phase

Using the [crawlers](https://docs.coveo.com/en/2159#modifying-a-crawling-root) configured on the search index, Sitecore iterates over a set of items and passes them to Coveo for Sitecore.
In most cases, when a document isn't indexed, the reason is that [Coveo for Sitecore hasn't received the item from Sitecore](https://docs.coveo.com/en/2558/).

Coveo for Sitecore creates [batches of 10 documents that are uploaded](https://docs.coveo.com/en/90/) in Coveo.
The time required to upload all items depends on the number of items, the time to [render the page](https://docs.coveo.com/en/2326/), and the bandwidth of your Internet connection.

### ![Number four icon for Documents Validation phase | Coveo](c4sc-v5/four.svg) Documents Validation phase

The `Waiting for Items to Be Searchable` step validates that all items sent to Coveo are being committed and are searchable.
It displays the number of committed items and the number of expected items.

A rebuild operation replaces the source content with a new set of items.
However, the old items have to be removed.
The `Removing Old Items` step monitors the old items and ensures they're removed.
This process times out after 5 minutes of inactivity.

> **Note**
>
> The Documents Validation phase waits for all steps in previous phases to complete.
> For example:
> 
> * The Documents Validation phase pauses while document uploading steps are still in progress.
> 
> * When you create a Coveo organization, the Coveo service has to deploy some components before the organization is fully functional.
> This process is called [organization provisioning](https://docs.coveo.com/en/2547/).
> Organization provisioning must be completed before steps in the Documents Validation phase can begin.

## Coveo rebuild logs

By default, the Coveo rebuild logs are recorded in `log.<DATE>.txt` and `log.<DATE>.<TIME>.txt` files in the `<SITECORE_INSTANCE_ROOT>\Data\logs` folder, with `INFO` as the log level.

> **Leading practice**
>
> The Coveo Diagnostic Page provides a convenient [log viewer](https://docs.coveo.com/en/2537#coveo-for-sitecore-log-viewer).
> It reads the `.txt` files from the `<SITECORE_INSTANCE_ROOT>\Data\logs` folder and color codes messages based on their severity.

Coveo rebuild log messages contain the name of the related source, which reduces confusion when many search indexes are rebuilt at the same time.
Coveo logs messages to mark the beginning and end of each rebuild phase.

![Image of Coveo rebuild logs | Coveo for Sitecore 5](https://docs.coveo.com/en/assets/images/c4sc-v5/coveo-rebuild-logs.png)
Color coded example of Coveo rebuild logs.  
![Number one icon for Setup Requirements phase | Coveo](c4sc-v5/one.svg) [Setup Requirements phase](#setup-requirements-phase)  
![One](c4sc-v5/two.svg) [Sending Permissions phase](#sending-permissions-phase)  
![One](c4sc-v5/three.svg) [Sending Documents phase](#sending-documents-phase)  
![One](c4sc-v5/four.svg) [Documents Validation phase](#documents-validation-phase)  

## Configuring custom Coveo logs

Coveo for Sitecore provides a [log4net custom log configuration](https://sitecore-community.github.io/docs/documentation/Sitecore%20Fundamentals/Logging/#custom_logs) that you can enable and customize.
This lets you:

* Centralize Coveo logs in separate files.

> **Note**
>
> By default, logs are recorded in `Coveo.Search.<DATE>.txt` files.

* Lower the log level to `DEBUG` to increase logging granularity.
Setting the log level to `DEBUG` gives you the ability to search for a Sitecore `Item ID` and better follow how the item is handled during a rebuild or at query time.

![Image of custom Coveo rebuild logs on the Diagnostic Page log viewer](https://docs.coveo.com/en/assets/images/c4sc-v5/custom-logs-on-debug-level.png)

To record Coveo logs in separate files and set the log level to `DEBUG`

. Open file `Coveo.SearchProvider.Custom.config` in a text editor.

> **Note**
>
> The `Coveo.SearchProvider.Custom.config` file is located in the `<SITECORE_INSTANCE_ROOT>\App_Config\Include\Coveo\` folder.

. Locate the `<log4net>` element.
It should be near the end of the file and commented out.

```xml
<!--
  <log4net>
    <appender name="CoveoLogger" type="log4net.Appender.RollingFileAppender, Sitecore.Logging">
      <file value="$(dataFolder)/logs/Coveo.Search" />
      <appendToFile value="true" />
      <rollingStyle value="Composite" />
      <datePattern value=".yyyyMMdd'.txt'" />
      <staticLogFileName value="false" />
      <maxSizeRollBackups value="20" />
      <maximumFileSize value="250MB" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%4t %d{ABSOLUTE} %l %-5p %m%n" />
      </layout>
    </appender>
    <logger name="Coveo" additivity="false">
      <level value="INFO" />
      <appender-ref ref="CoveoLogger" />
    </logger>
  </log4net>
-->
```

. Uncomment the `<log4net>` element.

. Set the log level to `DEBUG`.

```xml
<level value="DEBUG" />
```

> **Important**
>
> Setting the log level to `DEBUG` has an impact on server performance and the size of log files.
> Don't forget to set the log level back to `INFO` or to comment out the `<log4net>` element when you no longer need the added granularity.

. Save your changes.