Re:Ask HN: Is there a great open source crawler?
by ericwaller on Wednesday June 02, 2010 @11:22PM
(#1400297)
Scrapy (http://scrapy.org/) matches your description pretty well. You can specify which urls to crawl with regular expressions and then provide a bit of code to do some data extraction.
[ Parent
| on HN ]
There are no comments on this comment yet.