Crawler is a simple Web crawler (or spider). Its main goal is to abstract the boring and error-prone code from your codebase and let you focus on crawling the site. Also, it's quite easy to use.
Information regarding Project Releases and Project Resources. Note that the information here is a quote from Freecode.com page, and the downloads themselves may not be hosted on OSDN.
