Inspect Relevance Generative Answering (RGA) chunks

This is for:

Developer System Administrator
In this article
Important

The Coveo Knowledge Hub and Chunk Inspector are currently available as a beta to early-access customers only. Contact your Customer Success Manager for early access to this feature.

Inspecting the chunks that are used by RGA is an important step in the Answer evaluation and management flow. It provides you with insight that you can use to make modifications when managing your RGA answers.

Chunk Inspector answer | Coveo

You can use the Chunk Inspector in the Coveo Knowledge Hub to do the following:

  • See the segments of text (chunks) from your dataset that were used by RGA to generate a specific answer. The Chunk Inspector shows the chunks that were identified as being the most relevant during second-stage content retrieval, and the chunks that were sent to the large language model (LLM) to generate the answer.

  • See how an item in your index is segmented into chunks by the RGA model when it creates the embeddings.

To inspect your RGA chunks

  1. In the Coveo Administration Console header, use the application picker to select Knowledge Hub.

    Application picker in the Administration Console | Coveo

  2. In the Knowledge Hub navigation menu, select Chunk Inspector.

  3. Select one of the following options:

    • To see the chunks that were used by RGA when generating an answer, use the Answer ID or Search ID option. These options let you see the chunks that were identified by the RGA model as being the most relevant during second-stage content retrieval, and the chunks that were sent to the LLM for answer generation:

      Important

      If your search interface is associated with an Answer Manager configuration, you must use the Answer ID option. If your search interface is not associated with an Answer Manager configuration, you must use the Search ID option.

      • Answer ID: Select this option to use the Answer ID of a generated answer from a search interface that’s associated with an Answer Manager configuration.

        To use this option, feedback must have been provided for the generated answer that you want to inspect, and the feedback must appear in the answer configuration. To find the Answer ID for a generated answer, refer to the evaluation details of the answer feedback.

      • Search ID: Select this option to use the search UID for a user query in a search interface that is not associated with an Answer Manager configuration.

        To find the search UID for a specific user query, use the Visit Browser page to filter the list of visits by User Query and date. Then, expand the visit event that corresponds to the generated answer to obtain the search UID.

    • To see the chunks that were extracted for an indexed item by an RGA model when it created the embeddings, use the Item ID option.

      Select this option to search for an item using the item unique ID. Since an organization can include more than one RGA model, you must select which RGA Model to target. To find the item unique ID of an indexed item, use the Content Browser to view the item’s properties.

  4. Click Inspect. The chunk details appear.

  5. Review the details:

    • For Answer ID and Search ID inspection methods, the results show the chunks that were retrieved during second-stage content retrieval.

      For each chunk, the details show the following:

      Chunk Inspector answer | Coveo

      • Item unique ID: The ID of the indexed item that contains the chunk.

      • Chunk score: A measure of the semantic similarity between the chunk and the user query, as determined by the RGA model. The value is based on the RGA model’s algorithm and a preconfigured range, which isn’t user-configurable.

        The RGA model uses the chunk score to rank the chunks based on relevance to the user query, with higher scores indicating higher relevance. In the context of the RGA answer-generation flow, only the most relevant chunks are sent to the large language model (LLM) for answer generation.

        Note

        The Chunk score value is not to be confused with the relevancy threshold value, which is a setting that determines how strict the RGA model is when considering chunks for answer generation based on relevancy. There is no direct correlation between the chunk score and the relevancy threshold value.

      • Sent to LLM: Indicates if the chunk was sent to the LLM for answer generation. Only the chunks that are sent to the LLM are considered by the LLM when generating the answer.

      • Cited: Indicates whether the chunk was used by the LLM to generate the answer. These correspond to the citations that appear for the generated answer in the RGA component.

    • For the Item ID inspection method, the results show the chunks that were extracted by the RGA model when creating the embeddings.

      Chunk Inspector item | Coveo

  6. If required, evaluate the feedback for your RGA generated answers, or make adjustments to improve the question-answer experience of your RGA implementation. For more information, see Evaluate and manage Relevance Generative Answering (RGA) answers.

Required privileges

The following table indicates the privileges required to access the Coveo Knowledge Hub, and view or edit elements of the Chunk Inspector and associated panels (see Manage privileges and Privilege reference).

Important

The Enable access level on the Chunk inspector domain allows the member to view all segments of text (chunks) used during answer generation, as well as the chunks for a given indexed item. This may include content to which the member may not have access to otherwise through the repository’s permission system.

Exercise caution when granting this privilege to members.
Action Service - Domain Required access level

Access the Coveo Knowledge Hub

Knowledge - Knowledge hub

Enable

Organization - Organization

View

Access the Chunk Inspector

Knowledge - Chunk inspector

Enable

Select a model when using the Item ID inspection method

Machine Learning - Models

View