News for hackers. Stuff that matters.
Re:Ask YC: Where to start with creating a distributed crawler by inovica on Sunday March 30, 2008 @12:26PM (#150185)
Good response. We've created a basic crawler in Python, but are looking for something more powerful too. Heritrix above looks good

There are no comments on this comment yet.