Web scraping is a Sitemap and Web source configuration option that lets administrators exclude irrelevant sections, create metadata, and generate sub-items when indexing web pages.
For more information, see: