Description: heritrix is an open source network reptiles/Web Spiders, heritrix purpose is to track the page url to the expansion of the crawl, and finally for the search engine provides a wide range of data sources.
File list (Check if you may need any files):
Download