News for hackers. Stuff that matters.
Re:Ask HN: Open source focused crawler? by techaddict009 on Sunday February 09, 2014 @04:45AM (#7205001)
Check this out : http://commoncrawl.org/

Its not exactly what you are looking for but might help you.

There are no comments on this comment yet.