View Search Agent performance reports

This is for:

Developer System Administrator
Important

Coveo Conversational Search is a paid product extension that enables the Coveo Search Agent. Contact Coveo Sales or your Account Manager to add Coveo Conversational Search to your organization license.

To track the usage of the Search Agent, Coveo Analytics logs events for actions performed on the Search Agent component in your search interface.

This article explains how to create a report that helps you analyze the performance of your Search Agent implementation and the conversational search experience it delivers.

Tip

For complete implementation steps for the Coveo Search Agent, see Search Agent implementation overview.

To view Search Agent event data in a report, you must:

  1. Add custom dimensions to your Coveo organization to log the events.

  2. Create the report using the preconfigured Search Agent Performance dashboard template.

Add custom dimensions

To view data on the Search Agent custom events in a report, you must add the following custom dimensions to your Coveo organization:

  • c_answergenerated

  • c_conversationid

  • c_generativequestionansweringid

To add the custom dimensions for Search Agent reports

Add a custom dimension for each of the dimensions listed above using the following criteria:

  • For each of the custom dimensions, API name is the name of the dimension. For example, when creating the c_answergenerated dimension, the API name is c_answergenerated.

  • For each of the custom dimensions, Type must be Text.

  • For each of the custom dimensions, Related events must be set to Custom event.

Example

To create the c_answergenerated dimension, the settings are as follows:

Search Agent custom dimension | Coveo

Create a Search Agent report

You can quickly create a report to analyze the performance of your Search Agent implementation by using the preconfigured Search Agent Performance dashboard template.

This template provides a convenient way to build a report that surfaces key Search Agent metrics and insights related to your conversational search experience, helping you evaluate user engagement, answer rate, and business impact.

The template creates a report with three tabs:

  • Search Metrics: Contains report cards that show metrics such as the number of queries with and without generated answers (conversations), click-through rate, and average number of follow-up queries per conversation.

  • User Feedback: Displays user feedback data collected through the Search Agent component.

  • Self Service: Lets you see the impact of Search Agent’s answer generation on case submission, and by extension on self-service success, if you’re logging a custom event when a user submits a case during the same visit when accessing the Search Agent’s associated search interface.

To create a Search Agent report

  1. If you haven’t done so already, add the custom dimensions to your Coveo organization.

  2. Add a report dashboard using the preconfigured Search Agent Performance template.

Search Agent analytics events

This section lists the custom Coveo Analytics events that the Search Agent logs for the actions that users perform on the Search Agent component.

Note

In a conversational, multi-turn experience, the Search Agent also includes a conversationId value in the customData of some of the events below to identify the conversation that the interaction belongs to. This value is included in the generatedAnswerStreamEnd, openGeneratedAnswerSource, generatedAnswerSourceHover, likeGeneratedAnswer, dislikeGeneratedAnswer, and generatedAnswerCopyToClipboard events.
Value Event

  • eventType: generatedAnswer

  • eventValue: generatedAnswerStreamEnd

The answer was successfully streamed in the search interface. This event won’t be logged if the user initiates another search while the answer is being generated.

In customData:
answerGenerated: true

An answer was successfully generated for the query.

In customData:
answerGenerated: false

An answer wasn’t generated for the query because there wasn’t enough relevant information to generate an answer.

Note

If an answer wasn’t generated due to an empty query, empty search results, or an error, no event is logged.

  • eventType: generatedAnswer

  • eventValue: openGeneratedAnswerSource

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

    • citationId: The ID of the citation that was clicked.

    • permanentId: The permanent ID of the item corresponding to the citation that was clicked.

A citation link was clicked.

Note

As of the Coveo Headless v3 release, citation clicks are tracked as regular click events instead of custom click events. As a result, citation click events now have a click rank value of 1. Additionally, the click Event Cause value is set to generatedAnswerCitationClick.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerSourceHover

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

    • citationId: The ID of the citation that was clicked.

    • permanentId: The permanent ID of the item corresponding to the citation that was clicked.

    • citationHoverTimeMs: The duration of the hover in milliseconds (ms).

A user hovered over a citation link.

  • eventType: generatedAnswer

  • eventValue: likeGeneratedAnswer

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user clicked the thumbs-up icon to provide positive feedback for the generated answer.

  • eventType: generatedAnswer

  • eventValue: dislikeGeneratedAnswer

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user clicked the thumbs-down icon to provide negative feedback for the generated answer.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerCopyToClipboard

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user clicked the copy button to copy the generated answer to their clipboard.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerHideAnswers

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user closed (hid) the Search Agent component.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerShowAnswers

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user opened (showed) the Search Agent component.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerExpand

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user expanded the generated answer to view it in full.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerCollapse

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user collapsed the generated answer.

Inspect your search page

Use your browser developer tools to retrieve what’s being tracked:

  1. Open a Coveo-powered search interface that uses the Search Agent.

  2. Open your web browser’s developer tools.

    Note

    This article uses the Google Chrome developer tools. For browser-specific information, see:

  3. Select the Network tab.

  4. On the search page, perform a query that will generate an answer.

  5. Perform an action in the Search Agent component.

  6. Back in the developer tools window, under the Name column, click the latest custom call to Coveo Analytics.

  7. Select the Payload tab. You should now see the tracked Coveo Analytics events.