Index SAP Commerce Cloud

Ecommerce Developer Product Documentation

In this article

Prerequisites
Pull vs push
Pull implementation
Push implementation
What’s next?

This article explains how to index SAP Commerce Cloud ^[1] catalog by using either a pull or push approach. The indexed data can be used in a Coveo-powered search interface built with the SAP Commerce Cloud composable storefront (SAP Spartacus).

Note

The examples provided in this article are for a simple use case that involves products only.

Your final structure will vary depending on how you set up your catalog within SAP.

Prerequisites

To index your commerce content, you need a Coveo organization. Coveo lets you create a source which is the bridge to your commerce data.

If you have never used the Cloud Platform before, log in now.

Notes

Contact your sales representative to enable Coveo for Commerce features in your organization.
You can review your Coveo organization license limits.
Ensure that you set the Required privileges for your commerce organization.

Pull vs push

Depending on your needs and SAP Commerce configuration, one of the approaches will be more suitable for you.

The following table presents pros and cons of these indexing methods.

	Pull	Push
🟢	Partially takes care of indexing logic. Simplifies data mapping.	Doesn’t require Integration Objects. SOLR has predefined Resolvers and Providers suitable to implement indexing logic.
❌	Requires the use of Integration Objects	Requires custom Java code to work with the Coveo Stream API

Pull

Push

🟢

Partially takes care of indexing logic.

Simplifies data mapping.

Doesn’t require Integration Objects.

SOLR has predefined Resolvers and Providers suitable to implement indexing logic.

❌

Requires the use of Integration Objects

Requires custom Java code to work with the Coveo Stream API

See the dedicated sections below for details and required steps.

Pull: pros and cons

The pull approach implies usage of the Coveo SAP connector that maps data being sent from SAP Commerce to your Coveo organization.

However, this approach requires adjusting the configuration on SAP’s end

creating Integration Objects, which may be non-trivial if you’re unfamiliar with these objects
creating new scripts to calculate computed attributes. Doing so may mean rewriting business logic that is already present in a Solr provider

Push: pros and cons

The push approach is mainly handled on SAP’s end and does not require Integration Objects but creating a new Provider and Exporter instead.

However, this approach requires you to take care of mapping the data being sent from SAP Commerce to your Coveo organization. This micromanagement is the downside of this approach.

Pull implementation

Step 1: Expose your SAP catalog

SAP Commerce Cloud’s Integration API Module exposes a set of interfaces used for data integration. You’ll need to create a Product Integration Object to expose your products. Here are a few leading practices to consider when creating the product integration object:

Expose as many of your product root attributes as possible.
Ensure you add the Coveo IP address to the allowlist for your region.

To get OData2 representation through this API, see The OData2 GET API.

This is an example of what Coveo receives when calling the API:

{
  "d": {
    "results": [
        {
            "__metadata": {
                "id": "https://api.acmestore.com/odata2webservices/CoveoInboundProduct/Products",
                "uri": "https://api.acmestore.com/odata2webservices/CoveoInboundProduct/Products",
                "type": "HybrisCommerceOData.Product"
                },
            "code": "137220",
            "name": "Camcordertape DV 60min (2)",
            "manufacturerAID": "2DVM60PR-BT",
            "manufacturerName": "Sony",
            "case_color": "red",
            "description": "<br/><br/>Perfect digital picture with a resolution that exceeds 500 lines.<br/><br/>Infinite range of possibilities to edit and share your videos.<br/><br/>Possibility to record up to 50% more in LP mode.<br/><br/>Cleaning Tape: DVM12CLD.<br/><br/>",

…

            "integrationKey": "Online|electronicsProductCatalog|137220",
            "stockLevels": {
                "__deferred": {
                "uri": "https://api.acmestore.com/odata2webservices/CoveoInboundProduct/Products"
                }
            },
...
        }
    ],
"__next": "https://api.acmestore.com/odata2webservices/CoveoInboundProduct/Products?$filter=catalogVersion%2FintegrationKey%20eq%20%27Online%7CelectronicsProductCatalog%27&$top=1&$skiptoken=2"
    }
}
//...

Step 2: Fetch catalog content in Coveo

To index data accessible through an exposed REST API, you must create an SAP source. The attributes specified in the REST configuration ensure that the corresponding fields will appear in your indexed catalog.

While creating an SAP source, make sure to

Complete the authentication section for the method used by the API you want to retrieve content from.
Create fields in the index for each metadata you want to retrieve with the SAP source.
Map your source with all the fields you want to populate. If a field doesn’t have a mapping, you must create one.

You can adapt the values based on your API response, as each use case will be different.

{
  "services": [
    {
      "url": "https://api.acmestore.com/odata2webservices", 
      "headers": {
        "Accept": "application/json",
        "Application-Interface-Key": "xxxxxxxx" 
      },
      "paging": {
        "pageSize": 10,
        "offsetStart": 0,
        "offsetType": "item",
        "nextPageKey": "__next",
        "parameters": {
          "limit": "$top",
          "offset": "$skip"
        }
      },
      "endpoints": [ 
        {
          "path": "/Product/Products", 
          "method": "GET",
          "queryParameters": {
            "$filter": "catalogVersion/integrationKey eq 'Online%7CpowertoolsProductCatalog' and approvalStatus/code eq 'approved'",
            
            "$expand": "supercategories,thumbnails,picture,thumbnail,unit,catalogVersion/catalog,europe1Prices,europe1Prices/currency,bundleTemplates,localizedAttributes,supercategories/localizedAttributes,approvalStatus"
            
          },
          "itemPath": "d.results", 
          "itemType": "product", 
          "uri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]",
          
          "clickableUri": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]",
          "metadata": { 
            "objecttype": "Product", 
            "ec_brand": "%[brand]",
            "ec_name": "%[name]",
            "name": "%[name]",
            "ec_item_group_id": "%[code]",
            "ec_shortdesc": "%[summary]",
            "ec_description": "%[description]",
            "ec_price": "%[europe1Prices.results[0].price]",
            "ec_thumbnails": "https://accstorefront.commerce.ondemand.com%[thumbnail.URL]",
            "ec_in_stock": "%[stock]",
            "ec_images": "https://accstorefront.commerce.ondemand.com%[picture.URL]",
            "ec_category": "%[supercategories.results[:].name]",
            "ec_pricerange": "%[priceRange]",
            "ec_configurable": "%[configurable]",
            "ec_code": "%[code]",
            "ec_url": "https://accstorefront.commerce.ondemand.com/coveob2bstorefront/powertools/en/USD/%[url]"
          }
        }
      ]
    }
  ]
}
//...

	`url`. The URL of your SAP Commerce Cloud OData2 GET API.
	`Application-Interface-Key`. A unique Application Interface Key (Application Interface Register, AIR). It defines Coveo as a partner of SAP and once set, it shouldn’t be changed or removed.
	`endpoints`. You can specify as many endpoints as you need. By default, the SAP source JSON configuration contains three endpoints, one for: product (e.g., `"path": "/Product/Products"`) variant (e.g., `"path": "FootWearVariant/GenericVariantProducts"`) availability (e.g., `"path": "/Availability/Warehouses"`) You can remove or add endpoints depending on your use case. In the example above, there’s only one endpoint for products.
	`path`. The path to your products. It’s made in the format of `/name of the integration object created in the Backoffice/the type of the object`+`s`
	`filter`. The expression that specifies the products to be indexed. Must be URL-encoded.
	`expand`. The list of entries associated with a single integration object or a collection of integration objects.
	`itemPath`. Specifies the object with results. Its value should be `d.results`, according to the response structure.
	`itemType`. The type of item that the resource contains. Coveo uses this data to organize the indexed content.
	`uri`. The URI of the product’s PDP page.
	`metadata`. Maps attributes from the integration object to the Coveo fields. Some ecommerce fields are created and mapped by default. You can add customs fields in the Coveo Administration Console (platform-ca \| platform-eu \| platform-au).
	`objecttype`. The mandatory field for indexing via the SAP source. It indicates whether the document is a product, variant, or availability.

Once you have rebuilt your source, you can inspect the items with the Content Browser (platform-ca | platform-eu | platform-au).

Push implementation

SAP Commerce uses the Solr Search engine for search, facets, PLP results sorting, boosting, and query suggestions. The facet search configuration can be done through the graphical interface of the Backoffice Administration Cockpit.

Advantages of using existing Solr cronjobs

SAP Commerce uses cronjobs to initially index products into Solr and update that index after.

To push products from SAP Commerce to Coveo, we strongly recommend using existing Solr cronjobs.

You can create your own, new cronjobs, but it would require you to:

rewrite all the queries used to index products
recreate all the business logic that is already present in value providers
add new attributes to the indexed products, which may require code changes

It all can be non-trivial to implement and the final solution may be more error-prone. This is why using the existing cronjobs is the best practice here.

Step 1: Create a source in your Coveo organization

To fetch SAP content in a push implementation, we recommend using a Catalog source. See Add or edit a Catalog source.

Step 2: Create an SAP extension

You need to create an SAP extension within which you will implement Coveo indexation logic. Refer to the SAP documentation to learn how to create a new extension in your SAP project.

Step 3: Create a Coveo Solr server mode

To push documents to Coveo, you need to add a new Solr Server Mode and switch to it.

To create a new mode

In your project folder, go to the extension folder, e.g., coveocore.
From the resources folder, open the xxxxx-items.xml file, e.g., coveocore-items.xml.

In the SolrServerModes enum, add the coveo value like shown in the example below:

<enumtypes>
  <enumtype generate="true" code="SolrServerModes" autocreate="false">
    <value code="coveo" />
  </enumtype>
</enumtypes>

Step 4: Make the Coveo Solr server mode available in the Administration Cockpit

To make a newly created Coveo Solr server mode available in the UI, you need to update the SolrServerMode enum.

In your extension folder, go to the resources folder.
Open the xxxxx-beans.xml file, e.g., coveocore-beans.xml.

Add a new server mode, Coveo.

<enum class="de.hybris.platform.solrfacetsearch.config.SolrServerMode">
  <!-- other values -->
  <value>Coveo</value>
</enum>

In your extension folder, open the extensioninfo.xml file.

Add the solrfacetsearch extension.

<requires-extension name="solrfacetsearch"/>

Step 5: Specify the Coveo Solr server mode

To activate the newly created Solr server mode

In the Administration Cockpit, go to the System → Search and Navigation → Solr Facet Search Configuration → Facet Search Configurations.
In the list of configurations, double-click the required configuration.
In the Search and Index Configuration section, double-click the Solr server configuration value.
In the Mode dropdown menu, select coveo.
Click Save at the top of the modal window.

steps to change the solr server mode in the SAP Commerce Administration Cockpit

Step 6: Create a search provider

In your extension folder, go to the resources folder.
Open the xxxxx-spring.xml file, e.g., coveocore-spring.xml.

Add a new search provider, coveoSearchProviderFactory.

<alias name="coveoSearchProviderFactory" alias="solrSearchProviderFactory" />
<bean id="coveoSearchProviderFactory"
      class="com.coveo.service.impl.CoveoSearchProviderFactory" parent="defaultSolrSearchProviderFactory">
  <property name="solrStandaloneSearchProvider" ref="solrStandaloneSearchProvider" />
  <property name="solrCloudSearchProvider" ref="solrCloudSearchProvider" />
  <property name="xmlExportSearchProvider" ref="xmlExportSearchProvider" />
</bean>

In your extension folder, go the src folder.
Navigate to your package folder, e.g., com.coveo.service.impl.

Create a CoveoSearchProviderFactory.java file with the following code:

import de.hybris.platform.solrfacetsearch.config.FacetSearchConfig;
import de.hybris.platform.solrfacetsearch.config.IndexedType;
import de.hybris.platform.solrfacetsearch.config.SolrServerMode;
import de.hybris.platform.solrfacetsearch.solr.SolrSearchProvider;
import de.hybris.platform.solrfacetsearch.solr.exceptions.SolrServiceException;
import de.hybris.platform.solrfacetsearch.solr.impl.DefaultSolrSearchProviderFactory;

public class CoveoSearchProviderFactory extends DefaultSolrSearchProviderFactory {
    @Override
    public SolrSearchProvider getSearchProvider(FacetSearchConfig facetSearchConfig, IndexedType indexedType) throws SolrServiceException {
        SolrServerMode mode = facetSearchConfig.getSolrConfig().getMode();
        if(mode.equals(SolrServerMode.COVEO)){
            return getXmlExportSearchProvider();
        }
        return super.getSearchProvider(facetSearchConfig, indexedType);
    }
}

Step 7: Create the exporter class

The new exporter class should allow SAP Commerce to:

convert Solr documents to Coveo documents
push documents to Coveo using the Stream methods of the Push API

To add a new exporter class, perform the following steps.

In your extension folder, go to the resources folder.
Open the xxxxx-spring.xml file, e.g., coveocore-spring.xml.

Add a new exporter class

<bean id="solr.exporter.coveo" class="com.coveo.service.impl.CoveoExporter" />

In your extension folder, go to src/com.coveo.service.impl folder.

Create a CoveoExporter.java file. Use the following code as a starting template; the actual implementation would depend on your particular project and configuration.

import de.hybris.platform.solrfacetsearch.indexer.exceptions.ExporterException;
import de.hybris.platform.solrfacetsearch.indexer.spi.Exporter;
import com.coveo.platformsdk.model.document.PushDocument;

public class CoveoExporter implements Exporter {

    private static final Gson gson = new Gson();

    @Override
    public void exportToUpdateIndex(Collection<SolrInputDocument> collection, FacetSearchConfig facetSearchConfig, IndexedType indexedType) throws ExporterException {
        Configuration configuration = getConfiguration("configuration.json");
        List<PushDocument> documents = new ArrayList<PushDocument>();
        try {
            collection.stream().forEach(solrInputDoc ->{

                // This method needs to
                // 1. convert the Solr document into the Coveo document
                // 2. open a stream for uploading this document to Coveo
                // 3. upload the document
                // 4. close the stream

                // Note: it's not recommended to convert the document in the same class.
                // Instead, you have to use the converters & populators design patterns.

            });
        } catch (Exception e) {
            log.error("error");
        }
    }
}

For more info about the Stream API, see:

Stream your catalog data to your source to learn about creating of a catalog.
How to update your catalog to learn about full and partial updates of the existing catalog.
Stream API reference.

To make things easier for you, our team is working on creating a Stream API Java client library.

This article will be updated once the client library is released.

What’s next?

Once you have the data in the index, create a Coveo commerce catalog to define the structure of your commerce-related items. In other words, you will establish the relationship between the items you’ve indexed in the previous step by identifying which items are products, variants, and availability channels.

If you have a way to extract your content and want to push it to Coveo instead, see Index commerce catalog content.

1. SAP Commerce Cloud previously called Hybris.