View Search Agent performance reports

This is for:

Developer System Administrator
Important

Coveo Conversational Search is a paid product extension that enables the Coveo Search Agent. Contact Coveo Sales or your Account Manager to add Coveo Conversational Search to your organization license.

To track the usage of the Search Agent, Coveo Analytics logs events for actions performed on the Search Agent component in your search interface.

This article explains how to create a report that helps you analyze the performance of your Search Agent implementation and the conversational search experience it delivers.

Tip

For complete implementation steps for the Coveo Search Agent, see Search Agent implementation overview.

To view Search Agent event data in a report, you must:

  1. Add custom dimensions to your Coveo organization to log the events.

  2. Create the report using the preconfigured Search Agent Performance dashboard template.

Prerequisites

Add custom dimensions

To view data on the Search Agent custom events in a report, you must add the following custom dimensions to your Coveo organization:

  • c_answergenerated

  • c_conversationid

  • c_generativequestionansweringid

To add the custom dimensions for Search Agent reports

Add a custom dimension for each of the dimensions listed above using the following criteria:

  • In the Display name field, enter a descriptive name for the dimension.

  • Leave the Semi-structured dimension checkbox unselected.

  • In the API name field, enter the name of the dimension you’re creating, that is, c_answergenerated, c_conversationid, or c_generativequestionansweringid.

  • Set Type to Text.

  • Under Related events, select the event categories that apply to the dimension you’re creating, as shown in the following table:

    Dimension Related events Resulting API name

    c_answergenerated

    Custom

    custom_events.c_answergenerated

    c_conversationid

    Search, Click, and Custom

    c_conversationid

    c_generativequestionansweringid

    Search, Click, and Custom

    c_generativequestionansweringid
    Important

    The selected event categories determine the resulting API name. When you select only one category, Coveo prefixes the API name with that category, as in custom_events.c_conversationid. When you select all three categories, the API name remains the value you entered, without a prefix, as in c_conversationid. The Search Agent Performance dashboard template queries c_conversationid and c_generativequestionansweringid without prefixes. Selecting only one event category for c_conversationid or c_generativequestionansweringid causes the report to fail with errors such as Metric(s) 'countDistinct(c_conversationid)' were not found.

    The following example shows the c_conversationid dimension, for which you select all three event categories:

    Search Agent custom dimension example | Coveo
Note

Once you create a dimension, you can only edit its display name. If you already created one of these dimensions with an incorrect Related events selection, you must delete the dimension and then create it again with the correct selection.

Create a Search Agent report

You can quickly create a report to analyze the performance of your Search Agent implementation by using the preconfigured Search Agent Performance dashboard template.

This template provides a convenient way to build a report that surfaces key Search Agent metrics and insights related to your conversational search experience, helping you evaluate user engagement, answer rate, and business impact.

The template creates a report with three tabs:

  • Search Metrics: Contains report cards that show metrics such as the number of queries with and without generated answers (conversations), click-through rate, and average number of follow-up queries per conversation.

  • User Feedback: Displays user feedback data collected through the Search Agent component.

  • Self Service: Lets you see the impact of Search Agent’s answer generation on case submission, and by extension on self-service success, if you’re logging a custom event when a user submits a case during the same visit when accessing the Search Agent’s associated search interface.

To create a Search Agent report

  1. If you haven’t done so already, add the custom dimensions to your Coveo organization.

  2. Add a report dashboard using the preconfigured Search Agent Performance template.

Note

A missing c_answergenerated dimension doesn’t cause a report error. Instead, the Visits with answer=true card in the report drops the filter condition that references the dimension and returns visit counts that aren’t filtered by whether an answer was generated. Make sure you’ve created all required dimensions as instructed before interpreting the report data.

Search Agent analytics events

This section lists the custom Coveo Analytics events that the Search Agent logs for the actions that users perform on the Search Agent component.

Note

In a conversational, multi-turn experience, the Search Agent also includes a conversationId value in the customData of some of the events below to identify the conversation that the interaction belongs to. This value is included in the generatedAnswerStreamEnd, openGeneratedAnswerSource, generatedAnswerSourceHover, likeGeneratedAnswer, dislikeGeneratedAnswer, and generatedAnswerCopyToClipboard events.
Value Event

  • eventType: generatedAnswer

  • eventValue: generatedAnswerStreamEnd

The answer was successfully streamed in the search interface. This event won’t be logged if the user initiates another search while the answer is being generated.

In customData:
answerGenerated: true

An answer was successfully generated for the query.

In customData:
answerGenerated: false

An answer wasn’t generated for the query because there wasn’t enough relevant information to generate an answer.

Note

If an answer wasn’t generated due to an empty query, empty search results, or an error, no event is logged.

  • eventType: generatedAnswer

  • eventValue: openGeneratedAnswerSource

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

    • citationId: The ID of the citation that was clicked.

    • permanentId: The permanent ID of the item corresponding to the citation that was clicked.

A citation link was clicked.

Note

As of the Coveo Headless v3 release, citation clicks are tracked as regular click events instead of custom click events. As a result, citation click events now have a click rank value of 1. Additionally, the click Event Cause value is set to generatedAnswerCitationClick.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerSourceHover

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

    • citationId: The ID of the citation that was clicked.

    • permanentId: The permanent ID of the item corresponding to the citation that was clicked.

    • citationHoverTimeMs: The duration of the hover in milliseconds (ms).

A user hovered over a citation link.

  • eventType: generatedAnswer

  • eventValue: likeGeneratedAnswer

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user clicked the thumbs-up icon to provide positive feedback for the generated answer.

  • eventType: generatedAnswer

  • eventValue: dislikeGeneratedAnswer

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user clicked the thumbs-down icon to provide negative feedback for the generated answer.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerCopyToClipboard

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user clicked the copy button to copy the generated answer to their clipboard.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerHideAnswers

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user closed (hid) the Search Agent component.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerShowAnswers

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user opened (showed) the Search Agent component.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerExpand

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user expanded the generated answer to view it in full.

  • eventType: generatedAnswer

  • eventValue: generatedAnswerCollapse

  • In customData:

    • generativeQuestionAnsweringID: The ID of the generated answer.

A user collapsed the generated answer.

Inspect your search page

Use your browser developer tools to retrieve what’s being tracked:

  1. Open a Coveo-powered search interface that uses the Search Agent.

  2. Open your web browser’s developer tools.

    Note

    This article uses the Google Chrome developer tools. For browser-specific information, see:

  3. Select the Network tab.

  4. On the search page, perform a query that will generate an answer.

  5. Perform an action in the Search Agent component.

  6. Back in the developer tools window, under the Name column, click the latest custom call to Coveo Analytics.

  7. Select the Payload tab. You should now see the tracked Coveo Analytics events.