Snowflake reader account FAQ

This is for:

Business Analyst System Administrator

This article contains a list of the most frequently asked questions regarding the Snowflake reader account.

General

When should I use the reader account instead of the Data Share feature?

The reader account is recommended if you don’t have a Snowflake account, as it gives you a direct view to the live data of your Coveo organization which is stored in the Snowflake database. It lets you query different data tables, configure other business intelligence (BI) solutions, and precludes you from having to export data to a CSV file.

You can also do all of this with Data Share, along with some additional capabilities, such as:

  • Consolidating all your organization data (sandbox and production) over a single database.

  • Customizing your user roles and warehouses.

You can use the Data Share feature if you have your own Snowflake account.

Do I need to use the Snowflake reader account for my reports?

While it’s possible to use the reader account to create your own reports, the Coveo Usage Analytics out-of-the-box (OOTB) reports provide most of the information required for reporting purposes.

The reader account is provided primarily for:

  • Business intelligence (BI) integrations.

  • Data exports to the client data lake.

  • Advanced analysis on specific Coveo Search requests and UA events.

What specific Snowflake edition is used by Coveo?

Coveo uses the Snowflake Business Critical Edition and Tri-Secret encryption of data (see Snowflake Editions).

Snowflake credits

What’s the Snowflake warehouse size?

Coveo provides one X-small warehouse via the reader account.

What’s the default number of Snowflake credits for my license?

Coveo assigns ten Snowflake credits per month to every client within their Coveo production organization, and assigns five credits per month to each client using a sandbox organization.

If you have a Coveo Enterprise license, you can be granted additional Snowflake credits by contacting Coveo Sales.

Can I purchase additional Snowflake credits?

Yes, you can purchase additional credits, though you may need to modify your license. We recommend that you reach out to your Coveo Sales representative.

What happens when my Snowflake credit limit is reached?

Your Snowflake queries will no longer run and will return an error until the credits are replenished in the following month.

How is the Snowflake credit calculated?

Coveo provides a one X-small warehouse via the reader account, with the warehouse set to shut down automatically after one minute of inactivity. The computing costs for credits are determined by Snowflake, not Coveo. Therefore, we recommend that you consult Understanding Compute Cost for more details.

What’s the Snowflake credit calculation per query?

The calculation is as follows:

1 Snowflake credit = 1 hour of data processing time.

Therefore, the default of ten Snowflake credits assigned per month equates to ten hours of data processing time.

Example

A single standalone query takes five minutes, this equates to a charge of six minutes by Snowflake, which will then be calculated as follows:

6 minutes/60 mins per hour = 0.1 Snowflake credit

Can I use Snowflake credits from the upcoming month for my current month?

No, the assigned Snowflake credits are for the current month. At the start of each month, the limit is reset to your monthly credit limit.

Will there be additional charges if I go over my monthly Snowflake credit limit?

No, because when the limit is reached, any subsequent Snowflake queries will fail until the limit is reset in the following month.

Do I need a Snowflake license to use the reader account?

No, the Snowflake credits and the reader account are provided as part of your Coveo license and no additional Snowflake license is required. If you want to use Data Share, then a Snowflake License is required.

Snowflake integration for data exports

What’s the best way to export Coveo UA data?

We offer different data export options so that you can access and leverage data based on your specific usage requirements.

  • The Data Share feature is recommended for Coveo’s large clients who have multiple Coveo-powered solutions and pass Coveo Usage Analytics data. Since you have direct access to your Coveo data along with the rest of your existing data in the data lake, data exports aren’t necessary.

  • The Snowflake reader account is recommended for Coveo’s large clients who don’t have a Snowflake license since they can also stream large amounts of data in a more efficient way using common database drivers. It’s ideal for large exports and ETL integrations.

  • Coveo’s own usage analytics (UA) data exports are suitable for clients with one or two Coveo-powered solutions and are passing Coveo Usage Analytics. However, it’s important to consider that data exports have limitations, especially regarding the amount of data that can be exported (see Limits).

Important

The Visits API should never be used for any data export as it’s designed to feed a visualization for the Visit Browser, and is not intended to export large amounts of data.

Snowflake reader script

Is there an example of how to extract UA Data with a single Snowflake reader export call?

Yes, the following article describes how Python can be used to automate the integration with your reader account: Use case: Retrieve Coveo UA data with the Python connector.

Can I see a sample of a Snowflake query?

Snowflake supports SQL queries, and you can run them directly via the Snowflake web interface:

select * from "COVEO"."ua"."searches_shared" limit 100;

For more information on building SQL queries to retrieve your data, see Build SQL queries in Snowflake.

Is there an organization that I can use to test the time and credits required to run my scripts?

No, Coveo doesn’t provide an organization for this work effort. You can, however, use your sandbox organization to understanding the time and credits required in different contexts.

We recommend that you monitor the time needed to complete a Snowflake query, and then that you optimize your usage to maximize the Snowflake credits of your production organization, which will contain your Coveo Usage Analytics data.

Can I see an example on full exports and delta times?

Yes, here’s an example:

A large client with over three years of Coveo Usage Analytics data exports 115M rows of search impressions (the largest table) in less than 20 minutes, using streaming logic to their Data Lake. This is equivalent to approximately 0.33 credits. They also do daily delta exports of Coveo Usage Analytics that take less than five minutes to complete.

Can I use a Fivetran integration?

Yes, in fact, a few Coveo clients have started using this integration as well.

What are the primary keys for each data table?

The primary keys for each table are the following:

Table Primary key

Searches_shared

search_id

Clicks_shared

click_id

Custom_events_shared

custom_event_id

  • The all_events_shared table is the union of the searches_shared, the click_share, and custom_events_shared tables.

  • Since the IDs should be UUIDs, a primary column can be defined as coalesce(search_id, click_id, custom_event_id) if needed.

    Note

    Coveo performs deduplication to remove duplicate events with the same ID to ensure data accuracy, though some duplicate events may still remain since the entirety of historical data isn’t covered (see Review data health).

Can I have WRITE access to the COVEO Snowflake database?

No, because the COVEO Snowflake database is created from a share. Furthermore:

  • There’s no PUBLIC schema.

  • The COVEO database is read-only.

  • The reader account has a CUSTOMER database with write access (see What is Restricted/Allowed in a Reader Account?).

  • There are limitations imposed by Snowflake for write activities.

Depending on the integration option (for example, Fivetran), a client can create, view, and reference the COVEO database content within the CUSTOMER database. However, any work is owned by the client.

What is the fastest way to stream data?

The data is clustered by date, so the fastest way to stream it into another system is to get a slice of the data based on the datetime column without trying to sort it.

Example

Each day, you export the data using a simple query defining date range, with no order, aggregation, distinct, nor limit. Snowflake then streams the data back to your application or platform, similar to other cloud storage solutions. As with all cloud solutions, the network plays an important role in the speed.

About the Snowflake database

How is Coveo Usage Analytics (UA) data stored within Snowflake?

Coveo stores Coveo Usage Analytics in Snowflake using micro-partition based on dates.

Note

The main Snowflake account stores the data while the reader account accesses the data, which is why you see the data through views rather than physical tables in the reader account.

Where can I find UA table field descriptions with examples?

The table fields relate to the Coveo UA events that are logged to a table and not specifically the table itself as this is just a container. For details on the various events that can be logged to Coveo, see Log usage analytics events.

Notes

Switch from the reader account to Data Share

Can I switch from the reader account to Data Share?

Yes, you can switch from the reader account to Data Share. To use the Data Share feature, you need to have your own Snowflake account. For more information, see Optimize data strategies with Snowflake.

Can I import my reader account worksheets to my new account?

While you can’t directly import your worksheets from the reader account to your new account, you can copy the contents of your worksheets and use them into your new account. For more information, see Sharing worksheets across accounts.

Security and data recovery

How does Coveo ensure that data stored in Snowflake is secured?

Coveo uses the Snowflake Business Critical Edition and tri-secret encryption of data.

Additional details

  • Each client has their own unique, secured, and separate virtual warehouse which is accessible from either the assigned reader account or Data Share (if a client has their own Snowflake license).

  • Reader accounts leverage HTTPS access.

  • As per Snowflake’s security requirements, the reader accounts are managed and owned by the client and secured by both a login (credentials set by the user) and an IP allowlist, thereby ensuring that the user can’t access Snowflake from the public internet. The client can assign and enforce a client VPN endpoint IP allowlist to ensure that their assigned users use the client-managed VPN tunnels.

    • We recommend users to enroll in MFA on initial access.

  • Reader accounts are assigned per Coveo organization and therefore Snowflake data won’t allow access to another organization.

  • Reader accounts provide READ access only to the client data.

  • Reader accounts, per Coveo organization, are managed by administration members only.

  • Reader account worksheets are owned and managed by the specific reader account owner. Queries and worksheets aren’t visible by other account holders unless the owner grants access.

    Note

    Worksheets and queries aren’t backed up by Coveo. Therefore, it’s the account owner’s responsibility to export and keep a backup of their queries to their specific client-authorized systems. It’s also their responsibility to adhere to client security processes.

  • All reader accounts are encrypted using an AWS KMS key.

  • All Snowflake reader account holders have the same single role. Coveo doesn’t provide the ability to manage security within a reader account, since this will conflict with the Coveo automation that’s in place to manage reader accounts.

  • Reader account activities are logged within Snowflake.

Which fields can contain sensitive data according to GDPR conformity?

By default, Coveo doesn’t process any data that falls into the special categories defined as sensitive by the GDPR. However, Coveo is a data-neutral and data-agnostic platform, and clients are in full control of the data that they choose to submit to Coveo.

The client solution controls which UA events are passed to Coveo and then stored within Snowflake and accessible from a reader account, from Coveo reports, and from the Visit Browser.

For more information on compliance, we recommend the following: Data privacy features and Compliance.

Notes

Why are IPs required for the Snowflake reader account?

This is part of Snowflake’s network policies and it serves to provide an additional layer of security and control. For more information, see Network Policies.

Can Snowflake reader accounts support DHCP-assigned IPs?

Yes, they do. By allowing DHCP-assigned IPs, Snowflake enables flexibility in network configurations (see Network Policies).

Example

You’re assigned the following DHCP-assigned IP address: 165.1.187.200, but you want to expand to support the 24 scope range, therefore you enter the following: 165.1.187.1/24. For more information, see Add or block IP addresses.

What’s Snowflake’s recovery plan?

Coveo and the Snowflake services it uses are hosted in the Amazon Web Services (AWS) North Virginia, Oregon, Ireland, and Australia regions, active-active on multiple AWS availability zones, each one distributed on multiple data centers. Each data center is engineered to be isolated from failure in other data centers for the highest fault tolerance and stability.

Coveo maintains a Disaster Recovery Plan (DRP) that defines the requirements for information security, and the recovery of the Coveo Hosted Services in case of a major outage of a whole primary region in AWS. This plan is tested at minimum once a year.

Snowflake’s main account falls under the same recovery plan as the rest of Coveo’s platform.