Description: java web crawler written in simple, by setting the seed page, you can crawl a website.
To Search:
File list (Check if you may need any files):
CrawlerTest\.classpath
...........\.myhibernatedata
...........\.project
...........\bin\crawler\test\Crawler.class
...........\...\.......\....\Crawlinginfo.class
...........\...\.......\....\Crawlinginfo.hbm.xml
...........\...\.......\....\DownLoadPages.class
...........\...\.......\....\FileDownLoader.class
...........\...\.......\....\LinkDB.class
...........\...\.......\....\Queue.class
...........\...\.......\....\test.class
...........\...\hibernate.cfg.xml
...........\...\log4j.properties
...........\lib\commons-codec-1.3.jar
...........\...\commons-httpclient-3.1.jar
...........\...\commons-logging-1.1.1.jar
...........\...\htmllexer.jar
...........\...\htmlparser.jar
...........\...\mysql-connector-java-5.0.5-bin.jar
...........\src\crawler\test\Crawler.java
...........\...\.......\....\Crawlinginfo.hbm.xml
...........\...\.......\....\Crawlinginfo.java
...........\...\.......\....\DownLoadPages.java
...........\...\.......\....\FileDownLoader.java
...........\...\.......\....\LinkDB.java
...........\...\.......\....\Queue.java
...........\...\.......\....\test.java
...........\...\hibernate.cfg.xml
...........\...\log4j.properties
...........\bin\crawler\test
...........\src\crawler\test
...........\bin\crawler
...........\src\crawler
...........\bin
...........\docs
...........\lib
...........\src
CrawlerTest
Download