THIS IS ARCHIVED DOCUMENTATION

Creating a Search Interface Optimized for WebCrawlers Using the Coveo for Sitecore Legacy Search UI Framework

This is for:

Developer System Administrator

Coveo for Sitecore 4.1 (November 2018)

Web crawlers lead search engines such as Google, Bing, and Yahoo! to find and index the content of websites. These crawlers process your website pages and look for hyperlinks embedded in them. This is how they can browse through an entire website (and even through a network of related websites) without necessarily having its sitemap. However, such crawlers often have a hard time with today’s most commonly used web technologies, such as JavaScript. This is why Coveo provides a search layout specially optimized for web crawlers.

Using the WebCrawler Device

By default, layouts and sublayouts are assigned to the Default device. This means that web pages are rendered identically no matter if a standard web browser (for example, Chrome) or a search engine crawler (for example, GoogleBot) accesses them.

To circumvent this problem, you can create your own WebCrawler device and assign the Search Layout WebCrawler to it. Alternatively, you can assign the layout to the Default device, which means that Coveo generates a stripped-down rendering of search results no matter what the current device is.

Using the WebCrawler Search Layout

To build a Coveo-Powered Search Page optimized for mobile devices, you can use the mobile layouts and sublayouts. Here are the required steps:

It’s assumed here that you have already created a Coveo-Powered Search Page and have inserted a few components for the Default device.

  1. In the Sitecore Content Editor, select your Coveo-Powered Search Page.
  2. Open its Layout Details (Presentation > Details).

  3. Next to the WebCrawler device, choose Edit.

  4. In the next screen (Device Editor), under Layout, select Search Layout WebCrawler.

  5. Under Controls, add this mandatory component, and add it to the coveo-search-webcrawler placeholder: Coveo Search WebCrawler (Sublayouts/Coveo/Coveo Search WebCrawler).

  6. Preview your Coveo-Powered Search Page using the Page Editor (Publish > Preview).
  7. In the Page Editor, choose the WebCrawler device.

  8. You will get a page with search results completely generated on the server side:

Customizing the WebCrawler Layouts and Sublayouts

You can customize webCrawler layouts and sublayouts the same way you can customize the Search Layout components.

  1. Duplicate the sublayout or rendering (component) that you’re interested in (see Duplicating the Coveo Search Component{.unresolved}).
  2. Customize the .ascx file associated with your component (or its .cshtml file if you’re working with MVC views).

Note that only the Search component is currently available with the web crawler layout.