News for hackers. Stuff that matters.

A comment on Ask YC: Where to start with creating a distributed crawler. Parent comment

Re:Ask YC: Where to start with creating a distributed crawler by sonink on Sunday March 30, 2008 @01:38PM (#150272)
Nutch is good and I would second it, but I would suggest to NOT build a crawler - its not trivial and inadvised in a startup, that is unless your startup is just about building a crawler.
[ Parent | on HN ]

There are no comments on this comment yet.