External search engine optimization (SEO)

This is for:

Developer System Administrator

When using Coveo to power your public website, it’s important to keep in mind how the content crawled from query-based listing pages appear in external search engines such as Google or Bing. This article provides a high-level overview of some leading practices to ensure that your content can be optimally indexed by external search engines (SEs).

External search engines (SEs) index the web at frequent intervals and rank pages based on content and confidence. To be ranked effectively, pages that must be indexed by external SEs should meet certain standards.

Commerce listing pages powered by the Coveo Platform are typically rich with content that should be indexed by external crawlers. Keep in mind that external SE crawlers trigger queries as well as many requests against your website to index your web pages. You may want to minimize the impact of these queries/requests. To learn how, see Exclude search interfaces from external search engines.

Note

Search-driven content that’s frequently changing such as search pages and recommendation interfaces is irrelevant from an SEO perspective and should therefore not be indexed by external SEs. Consider using static content along with dynamic content.

Server-side versus client-side rendering

It’s easier for common SEs to crawl and index server-side rendered (SSR) websites than client-side rendered (CSR) websites as the content is available to external SEs before it’s rendered on the end user’s browser.

Other strategies, such as dynamic pre-rendering or hybrid SSR/CSR solutions, can also be used to optimize the ranking of Coveo-powered search pages on external SEs.

While the Coveo Atomic and Coveo JavaScript Search Framework are pure client-side solutions, other options are available to build your search interface:

Coveo Headless

Coveo Headless is a library for developing your own Coveo-powered UI components. It wraps the complexity of the Coveo APIs while giving you full flexibility over your UI implementation. It’s used, for example, in Coveo Atomic, to handle application state and Coveo interactions.

Server-side rendering using the Coveo Search API

You can leverage the Coveo Search API directly to build your search interface from scratch. Since this approach gives you full control over your interface implementation, it allows you to implement server-side rendering.

Use the Coveo Atomic or JavaScript Search Framework with a pre-rendering tool

Since Coveo Atomic and Coveo JavaScript search frameworks are client-side rendering solutions, it’s not optimal from an SEO point of view.

If you have to use these frameworks to build your search interface, and need the interface to be indexed by external SEs, you can use a pre-rendering tool, such as Prerender, to improve the interface ranking on external SEs.

Stability

External SEs usually rank static pages higher than fully dynamic web pages. Whenever an external SE crawls a page, it expects the page to remain similar to the last time it was visited.

Due to their dynamic nature, Coveo-powered search pages that must be indexed and made searchable by external SEs may not rank effectively if some precautions are not taken.

Use static content along with dynamic content

Since external SEs seek stability for optimal ranking, a leading practice is to use both static and dynamic content on the pages that must be indexed by external SEs.

Example

On a Coveo-powered ecommerce website, you have a Gaming Computers listing page that incorporates product recommendation interfaces. These interfaces recommend items according to the current user’s profile and action history. Those recommendations are dynamic as they differ depending on the user who accesses the page. Since you want this listing page to rank well in external SEs, you incorporate a static interface that always shows the same products regardless of the user who browses the page. Doing so allows external SEs to recognize that some of the content remains the same every time it crawls the Gaming Computers listing page.

Use stable and readable URLs

When indexing content, external SEs expect URLs to be stable and well formatted. URLs typically follow the following format:

[protocol]://[hostName][urlPathname]#[urlHash]?[queryString]

Where:

  • protocol is the protocol that the browser uses to request the resource (for example, https). Note that external SEs usually give more importance to URLs that use the https protocol over http.

  • hostName is the host that holds the resource (for example, coveo.com).

  • urlPathname identifies the page location in the directory structure (for example, /en/search).

  • urlHash points the browser to a specific spot in a page or website (for example, #q=platform).

  • queryString assigns values to specified parameters (for example, ?q=platform).

Example
https://coveo.com/en/search/#q=platform/?q=platform

When indexing web pages, common external SEs ignore everything that comes after the # or ? URL characters. By default, Coveo-powered search pages (Coveo JavaScript Search Framework, Atomic, or Headless) put facet selections after the urlHash (#) to avoid external SEs indexing a new page for every facet combination.

Example

You have a search interface that can be accessed through the following URL:

https://www.acme.com/en/search

When a user selects the computer facet value of the productType facet, the URL becomes:

https://www.acme.com/en/search#f:productType=[computer]

Even though a user can type the computer URL to access a search page that only shows items whose productType is computer, this URL won’t be indexed by external SEs as they ignore everything that comes after the urlHash.

Use facet selections in the URL pathname

Important

Including facet selections in the URL pathname will cause all possible combinations of facet selections for the specified facets to be indexed by external SEs. This can cause external SEs to make a large number of requests, increasing the queries per month (QPM) count for the related Coveo organization (see What counts as a query).

To limit the requests made by external SEs in your commerce search interface, you can use canonical URLs to tell external SEs which URL should be indexed when multiple URLs lead to very similar content. For more information on canonical URLs, and how to configure them, see Consolidate duplicate URLs.

Since common external SEs ignore everything that comes after the # or ? URL characters when indexing web pages, this may cause issues if you want a Coveo-powered ecommerce search interface that contains specific facet selections to be crawled and indexed by external SEs.

You can make these facet selections indexable by external SEs by moving them directly in the URL pathname.

Example

On a Coveo-powered ecommerce website, you want certain facet selections to be indexed by external SEs to improve organic ranking when customers are looking for specific brands or product categories. Therefore, you set your interface to put facet selections for the brand and productCategory facets directly in the URL pathname.

To avoid having less important facet selections to be indexed by external SEs, you keep the default behavior for other facets, such as size and priceRange.

With this configuration, the following facet selections:

  • brand: Blurton

  • productCategory: Snowboard

  • size: 160

  • priceRange: 500-750

will generate the following URL:

https://www.acme.com/en/search/brand/blurton/productCategory/snowboard#f:size=[160]&f:priceRange=[500-750]

Use a sitemap

It’s a common SEO good practice to have a sitemap for your site. Using a sitemap ensures that your content is indexed by external SEs, hence the item directly related to an end user’s search will be returned rather than the search page that contains the item.

For more information on sitemaps and how to configure them, see Learn about sitemaps.