Web Scraping Configuration

A Web and Sitemap source can include a web scraping configuration, a powerful tool allowing you (when you have the required privileges ) to precisely select the web page content to index, exclude specific parts, extract content to create metadata, and create sub-items.

The web scraping configuration must however be specified in JSON format, thus requiring more technical skills. You must add the web scraping configuration to your Web or Sitemap source in JSON format (see Add/Edit Web Source - Panel or Adding a Sitemap Source).

Consider using the Coveo Labs Web Scraper Helper available on the Chrome Web Store and on GitHub (project called web-scraper-helper) to more easily create and especially test web scraping configurations.