Examining Usage Analytics Data

The Coveo Usage Analytics (Coveo UA) service allows you to visualize the usage analytics data of your search solution by generating meaningful dashboards and explorers. As such, this service provides essential feedback when designing and improving your search solution. This feedback allows you to know whether your solution is relevant, where it needs improvement, where business opportunities might lie, and so on.

Set Up

In order for the Usage Analytics service to be effective, your search interfaces need to log appropriate usage analytics events, which become dimensions and metrics, available in reports.

Log Usage Analytics Events

When creating a search interface using the Coveo JavaScript Search Framework, you can include an Analytics component for the framework to automatically log standard events for you, namely search, click, and custom events (see Include an Analytics Component). However, you may want to customize those standard events, perhaps by adding custom metadata and by sending custom context information (see, respectively, Sending Custom Metadata with Search, Click, or Custom Events and Sending Custom Context Information). It’s also possible to use the JavaScript Search Framework to log your very own search, click, and custom events (see Sending Your Own Search, Click, or Custom Events).

When you don’t use the JavaScript Search Framework, you need to do much more work, using the Coveo APIs directly (see Logging Usage Analytics Events).

You can use your browser console to track the events logged by your search interface.

You submit a query from a search interface, causing a search event to be logged. In your web browser, you open your developer tools to inspect the corresponding HTTP request, which looks like what follows.

Header

POST https://usageanalytics.coveo.com/rest/v15/analytics/searches?visitor=28s6g49d-f81s-1435-2r5x153dle72 HTTP/1.1
 
Content-Type: application/json
Cookie: visitor=28s6g49d-f81s-1435-2r5x153dle72
Authorization: Bearer **********-****-****-****-************

Payload

[
  {
    "anonymous": false,
    "language": "en",
    "originLevel1": "CommunitySearchPage",
    "originLevel2": "All",
    "actionCause": "searchboxSubmit",
    "queryText": "coveo",
    "responseTime": 145,
    "searchQueryUid": "7bfc652a-9dea-4811-b3f9-6d24345c37ce",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36"
  }
]

If you plan to leverage the Coveo Machine Learning (Coveo ML) Event Recommendations (ER) feature in your search solution, you should also log view events to feed your models (see Sending Usage Analytics View Events).

Dimensions and Metrics

The Usage Analytics service extracts the information logged in usage analytics events into dimensions and metrics (see Usage Analytics Dimensions and Usage Analytics Metrics). You can use these dimensions and metrics in reports.

To exploit custom metadata in your reports, you need to create corresponding dimensions (see Adding and Managing Dimensions on Custom Metadata).

Explore the Default Summary Dashboard

The Summary dashboard is a predefined general overview of all the basic information captured in usage analytics. This is a good out-of-the-box starting point to analyze your data (see Reviewing the Search Usage Trends From the Summary Dashboard).

Activity Tab

The Activity Tab is where you can see the number of queries, clicks, visits, and users captured over a given period of time. It provides a general idea of what happened and who used your Coveo interfaces.

Coveo solutions activity panel

The graph displays the query, click, visit, and user counts for the selected frequency and date range. Right next to the graph, you can see the total, average, and peak counts for the same metrics.

The two pie charts provide a breakdown of visits by country and device.

The tables list of the most popular tabs, queries, and items.

At the top right of the tab, you can find the date range. Clicking the date range allows you to choose the time period you want to analyze.

Keywords Tab

The Keywords tab lists keywords that are most often used in queries as well as trending keywords. It also plots out a graph of the number of unique keywords used over time. As with all reports in the dashboard, you can drill down to specific keywords, change the analysis time frame using the date range selection, and add new parameters to each table or graph to meet your needs.

Coveo solutions keywords panel

Relevancy Tab

The Relevancy tab graphs the average click-rank and the click-through over time, and also lists queries with the lowest relevance.

Coveo solutions relevancy panel

The click rank metric is the average position of the clicked item in the result list. The lower the value, the better. A value over 10 should be investigated.

The click-through graph shows the ratio of queries which are followed by a click from the end-user. The higher the value, the better. Typically, queries displaying optimal results have a click-through ratio above 0.66. Values under 0.50 should be investigated, particularly those which also have a high search event count (users frequently enter this query, but don’t click anything).

The Relevance Index is a derived metric based on the click-through ratio, the Average Click Rank, and the frequency of the query (see How Is the Relevance Index Calculated and How Should It be Used?). The higher the score, the better. The leading practice is to investigate anything at or below 0.50.

Content Gaps Tab

The Content Gaps tab analyzes the queries entered by users to identify the top occurrences with no results, no clicks, or poor ranking of clicked content. This analysis ultimately helps in identifying queries which don’t return relevant results and, from there, pinpoint potential content gaps (see Identifying Content Gaps).

Coveo solutions content gaps panel

Knowing the topics which are most searched by end users also helps in prioritizing content creation. The content gaps tab also displays time series graphs that show ​the number of total queries submitted​ and the number of queries that returned no results​. The objective is to see queries with no results decrease over time.

The number of search events without results may also include queries for which results exist, but that were not returned to the end user due to a lack of permission or over-filtering.

Create Custom Dashboards

The summary dashboard may not be sufficient for your needs. If that’s the case, you may need to build custom ones, perhaps leveraging you own custom dimensions (see Creating Usage Analytics Dashboards).

Coveo solutions custom panel

What’s Next?

The next article in this section outlines the various ways you can setup query pipelines in your Coveo organization to ensure that specific relevance tuning rules and Coveo ML models only apply to the proper audiences (see Setting up Your Query Pipeline).

Recommended Articles