News for hackers. Stuff that matters.
Re:Ask HN: Is there a great open source crawler? by yourabi on Wednesday June 02, 2010 @10:57PM (#1400252)
None of the ones I know are that simple - but Check out Heritrix at http://crawler.archive.org and Nutch at http://nutch.apache.org Also worth checking out is http://80legs.com

There are no comments on this comment yet.