Web scraping

Web scraping is a Sitemap and Web source configuration option that allows administrators to exclude irrelevant sections, create metadata, and generate sub-items when indexing web pages.