---
title: Indexing optimization with concurrency and batch size
slug: p41b1355
canonical_url: https://docs.coveo.com/en/p41b1355/
collection: coveo-for-commerce
source_format: adoc
---
# Indexing optimization with concurrency and batch size
Full indexing may be a time-consuming process, especially for large [catalog data](https://docs.coveo.com/en/obcf0333/).
To optimize the full indexing process, SAP Commerce Cloud allows for [concurrent indexing of multiple items](https://help.sap.com/docs/SAP_COMMERCE/d0224eca81e249cb821f2cdf45a82ace/8bf4a40e86691014a5b8ad934407c56e.html).

This article provides guidance on how to configure the concurrency and batch size settings to optimize the indexing process in SAP Commerce Cloud 2211.37 and later.

## Recommendations

SAP Commerce Cloud exposes a configuration object called `[SnIndexerConfiguration](https://help.sap.com/doc/a5a94e4606244fefbbfa6ca0fdeb1c99/2302/en-US/de/hybris/platform/searchservices/admin/data/SnIndexerConfiguration.html)` with the following properties:

[cols="2,5"]
|===
|Property |Description

a|`concurrency`
|Defines the maximum number of threads that can be used for indexing.

The default value is `5`.

a|`batchSize`
|Defines the number of items that can be processed in a single batch.

The default and minimum value is `100`.
|===

While the default values are suitable for most use cases, you can adjust them if you want to optimize the indexing process.
Consider the following recommendations:

* If the [catalog objects](https://docs.coveo.com/en/ncig0154/) have many attributes, they require as many value providers to be called.
This can make SAP Commerce Cloud spend more time on building processing each batch, so it's recommended to set the lower `batchSize` value.

* If the [catalog objects](https://docs.coveo.com/en/ncig0154/) have few attributes, increasing the `batchSize` value can help to speed up the indexing process.

* If the [catalog objects](https://docs.coveo.com/en/ncig0154/) have few attributes and the [catalog data](https://docs.coveo.com/en/obcf0333/) is large, you can increase both `concurrency` and `batchSize` values to speed up the indexing process.

> **Note**
>
> Such a configuration might require more processing power from the server.

> **Important**
>
> High concurrency and low batch size values can potentially hit the Stream API limits (see [Stream API limits (full updates)](https://docs.coveo.com/en/p4eb0129#stream-api-limits) or [Stream API limits (partial updates)](https://docs.coveo.com/en/p4eb0515#stream-api-limits)).
> If that happens, the indexing process will be interrupted and the retry mechanism will try to reprocess the failed items.

During indexing optimization, monitor your SAP Commerce Cloud server's performance, track failed indexing, and adjust the `SnIndexerConfiguration` values accordingly.

## Set up concurrency and batch size settings in the Backoffice

For the [required index type](https://help.sap.com/docs/SEARCH_SERVICE/96c6b036f78a4d4b99972cfaf09a008a/ce7178d8a00844c2aeddf98443882830.html), create a [new indexer configuration](https://help.sap.com/docs/SAP_COMMERCE_CLOUD_PUBLIC_CLOUD/aa417173fe4a4ba5a473c93eb730a417/d13867b756c5484b8db4a02785dfb743.html#procedure) with the following values:

[cols="2,5"]
|===
|**Identifier**
|Enter a unique identifier for a new configuration, for example, `coveoIndexerConfiguration`.

|**Name**
|Enter a name for a new search provider, for example, `Coveo Catalog Indexer Configuration`.

|**Concurrency**
|Enter the maximum number of threads that can be used for indexing. The default value is 5.

|**Batch size**
|Enter the number of items that can be processed in a single batch. The default value is 100.
|===

The configuration is created and attached to your full indexer, you can now run it with the new settings applied.
See [Run the indexers](https://docs.coveo.com/en/ladf2011#step-12-run-the-indexers).

## Set up concurrency and batch size settings in the impex file

You can also set up the concurrency and batch size settings in the impex file.
The following example repeats the configuration shown in the previous section.

```xml
INSERT_UPDATE SnIndexerConfiguration; &indexerConfig            ; id[unique = true]         ; name                                    ; concurrency ; batchSize
                                    ; coveoIndexerConfiguration ; coveoIndexerConfiguration ; Coveo Electronics Indexer Configuration ; 5           ; 100

INSERT_UPDATE SnIndexType; indexConfiguration(id)   ; id[unique = true]      ; name                     ; itemComposedType(code) ; identityProvider   ; listeners                            ; catalogs(id)              ; stores(uid) ; indexerConfiguration(&indexerConfig)
                         ; $coveoIndexConfiguration ; $coveoProductIndexType ; CoveoElectronics Product ; Product                ; snIdentityProvider ; catalogVersionFilterSnSearchListener ; electronicsProductCatalog ; electronics ; coveoIndexerConfiguration
```