Introduction - If you have any usage issues, please Google them yourself
Likened to a spider web of the Internet, so Spider spider is crawling around on the Internet.
Web spider is to find the page through a web link address.
Starting a one page website (usually home), read the content of the page to find other links on the page in the address,
Then look through these links address of the next page, this has been the cycle continues, until all the pages of this site are crawled last.
If the entire Internet as a website, so web spiders can use this principle to all the pages on the Internet you have to crawl down.
It would appear that a spider web crawler, a program to crawl pages.