---
title: Catalog enrichment using image classification
slug: o43a2025
canonical_url: https://docs.coveo.com/en/o43a2025/
collection: coveo-for-commerce
source_format: adoc
---
# Catalog enrichment using image classification

Building product content usually involves a lot of manual work, which is time-consuming and error-prone.
Traditionally, this entails meticulously inputting relevant [metadata](https://docs.coveo.com/en/218/) for each product, leading to potential gaps or discrepancies in product information.
For example, if you're selling a product, you may need to manually add additional product information, such as the product `dimensions`, `color`, or `manufacturer`.
This process can be tedious, especially if you have a large inventory of products.

Image classification harnesses the power of AI to automatically enrich product content with relevant [metadata](https://docs.coveo.com/en/218/) using product images.
This process helps accurately describe your products, improving the relevance of your search experience, product recommendations, and shopper personalization.

Among other benefits, image classification offers the ability to:

* Fill missing product information that's not currently available in your product content.

* Ensure consistency in your product [metadata](https://docs.coveo.com/en/218/).
For example, without image classification, some products might have their color described as `blue` while others might have it described as `navy`.

* Improve the quality of the search results.

* Increase the number of results matching a search [query](https://docs.coveo.com/en/231/).

* Reduce the amount of manual work required to build your product content.

* Focus on filling retailer-specific information, such as pricing and availability.

![Enriching a product catalog with image classification](https://docs.coveo.com/en/assets/images/coveo-for-commerce/images/kayak-catalog-enrichment.png)

## Image classification at Coveo

To enrich product content through image classification, you'll need to rely on a third-party image classification service.

Generic image classification services, such as [Google Cloud Vision](https://cloud.google.com/vision?hl=en) or [Amazon Rekognition](https://aws.amazon.com/rekognition/), can be used to classify a wide range of images, but they might not be as accurate as specialized services, which are designed to classify images in specific product verticals, such as fashion retail or home furniture.

For example, if you operate a fashion retail business, you might want to use a specialized image classification service that's designed to classify clothing items.

### Prerequisites

Before you can start using image classification to enrich your product content and push the [metadata](https://docs.coveo.com/en/218/) to the [Coveo Platform](https://docs.coveo.com/en/186/), you need:

* Access to an image classification service, such as [Google Cloud Vision](https://cloud.google.com/vision?hl=en) or [Amazon Rekognition](https://aws.amazon.com/rekognition/).

* A [source](https://docs.coveo.com/en/246/) that [indexes](https://docs.coveo.com/en/204/) your product content, such as a [Catalog source](https://docs.coveo.com/en/l5if0244/).

### How to enrich your product content with image classification

In your own ecosystem, your content usually resides in a storage system, such as a product information management (PIM) or digital asset management (DAM).
In a typical commerce scenario, you must develop middleware that connects your storage system to the Coveo [index](https://docs.coveo.com/en/204/), formatting and sending data via the [Coveo Stream API](https://docs.coveo.com/en/p48b0322/).

The product content must be formatted as a JSON object which contains the [metadata](https://docs.coveo.com/en/218/) that's relevant to the [item](https://docs.coveo.com/en/pa8f6515/) [object type](https://docs.coveo.com/en/n8of5561/) you're [indexing](https://docs.coveo.com/en/204/).

The following diagram shows the typical flow of [indexing](https://docs.coveo.com/en/204/) product content to a Coveo [source](https://docs.coveo.com/en/246/) when you aren't leveraging image classification to enrich your product content:

![Typical flow of indexing product content to Coveo](https://docs.coveo.com/en/assets/images/coveo-for-commerce/images/catalog-enrichment-process-simple-flow.png)

[cols="2"]
|===
|1
|The content stored in your content storage systems is sent to a middleware.

|2
|The middleware formats the content in a JSON file.

|3
|The middleware sends the JSON file to the Coveo Stream API.

|4
|The Coveo Stream API pushes the content to your [source](https://docs.coveo.com/en/246/) in the Coveo [index](https://docs.coveo.com/en/204/).
|===

When you're enriching your product content with image classification, you'll need to add the [metadata](https://docs.coveo.com/en/218/) returned by the image classification service to your product content.

Your product content usually includes images, which are formatted as URLs that give public access to the image files, typically stored in a cloud storage service.

Most image classification services provide an API that you can use to send an image URL and get back the classification results.
You'll need to use the response from the image classification service to enrich your product content with the relevant [metadata](https://docs.coveo.com/en/218/).

For example, by sending the [kayak image](https://images.barca.group/Sports/mj/Canoes%20and%20Kayaks/Kayaks/Sports%20Kayaks/38_Yellow_Carbon-Kevlar/5b284698c6e0_bottom_right.webp) to an image classification service, you might get back the following response:

```json
{
  "kayak_type": "single seat",
  "color": "Yellow",
  "accessories": ["Water proof front storage", "Back storage"],
  [...]
}
```

#### Send enriched content to Coveo

Once you've received the [metadata](https://docs.coveo.com/en/218/) from the image classification service, you must add it to your product content and push it to your Coveo [index](https://docs.coveo.com/en/204/).

The process of pushing the enriched product content to the [Coveo Platform](https://docs.coveo.com/en/186/) is the same, whether you push the entire product inventory or only the updated products.
In both cases, Coveo recommends that the enrichment procedure is done at the content storage system level, before the content is pushed to the [Coveo Platform](https://docs.coveo.com/en/186/).

Once your product content includes the [metadata](https://docs.coveo.com/en/218/) returned by the image classification service, you can push it to your Coveo [index](https://docs.coveo.com/en/204/) using the [Coveo Stream API](https://docs.coveo.com/en/p48b0322/).
If your [source](https://docs.coveo.com/en/246/) already contains your product content, you can't push only the updated [metadata](https://docs.coveo.com/en/218/) to the [Coveo Platform](https://docs.coveo.com/en/186/).
To update your [source](https://docs.coveo.com/en/246/) with the new [metadata](https://docs.coveo.com/en/218/) provided by the image classification service, you'll need to push your whole product inventory to the [Coveo Platform](https://docs.coveo.com/en/186/) again using the Coveo Stream API.

The following diagram shows the flow of enriching your product content with image classification and sending it to your [source](https://docs.coveo.com/en/246/) in the [Coveo Platform](https://docs.coveo.com/en/186/):

![Flow of enriching your product content with image classification](https://docs.coveo.com/en/assets/images/coveo-for-commerce/images/catalog-enrichment-process-enrichment-flow.png)

[cols="2"]
|===
|1
|URLs of the product images stored in your content storage system are sent to the image classification service.

|2
|The image classification service processes the images.

|3
|The image classification service sends back the [metadata](https://docs.coveo.com/en/218/) to your content storage system, where the classification results are added to the product content.

|4
|The content stored in your content storage systems is sent to a middleware.

|5
|The middleware formats the content to match Coveo's catalog schemas in a JSON file (see [Commerce fields](https://docs.coveo.com/en/n73f0502/)).

|6
|The middleware sends the JSON file to the Coveo Stream API.

|7
|The Coveo Stream API pushes the content to your [source](https://docs.coveo.com/en/246/) in the Coveo [index](https://docs.coveo.com/en/204/).
|===

## About fields and field mappings

When you're enriching your [catalog data](https://docs.coveo.com/en/obcf0333/) with image classifications, ensure that the [metadata](https://docs.coveo.com/en/218/) you're adding is correctly [mapped](https://docs.coveo.com/en/217/) to the [fields](https://docs.coveo.com/en/200/) in your Coveo [index](https://docs.coveo.com/en/204/).

**Example**

If the image classification service returns a `kayak_type` [metadata](https://docs.coveo.com/en/218/) value, and that you want this information to be used in the Coveo [index](https://docs.coveo.com/en/204/), you must:

. [Create](https://docs.coveo.com/en/n73f0502#create-additional-commerce-fields) a `kayak_type` [field](https://docs.coveo.com/en/200/) in your Coveo [index](https://docs.coveo.com/en/204/).
Ensure that these [fields](https://docs.coveo.com/en/200/) have the **Free text search** option enabled.
See [Add a field](https://docs.coveo.com/en/1833#add-a-field) for instructions on how to configure [fields](https://docs.coveo.com/en/200/) using the [Coveo Administration Console](https://docs.coveo.com/en/183/).

. [Map](https://docs.coveo.com/en/1640/) the `kayak_type` [metadata](https://docs.coveo.com/en/218/) to the `kayak_type` [field](https://docs.coveo.com/en/200/) in your Coveo [index](https://docs.coveo.com/en/204/).

## What's next?

* When you've enriched your product content with image classification and pushed the [metadata](https://docs.coveo.com/en/218/) to your Coveo [index](https://docs.coveo.com/en/204/), you can use this [metadata](https://docs.coveo.com/en/218/) to improve the relevance of your search results, recommendations, and shopper personalization.

To ensure that your product content now includes the [metadata](https://docs.coveo.com/en/218/) returned by the image classification service, you can use the [Content Browser](https://docs.coveo.com/en/2865/) to [inspect the content](https://docs.coveo.com/en/n8ge0096/) of your [source](https://docs.coveo.com/en/246/).

* When you've enriched your product content with image classification, you can leverage the classifications to implement [image-based search](https://docs.coveo.com/en/o5e97044/) on your Coveo-powered search interfaces.