1. Set the maximum number of pages to crawl. When the crawler reaches that limit, it will stop processing URLs. The other option is to set it to unlimited. You should note that it is always possible to click the stop button to manually stop the crawlers progress.
2. Set the maximum extraction depth. Use this option to narrow your crawler to a "shallow" search.
Back to Main