News for hackers. Stuff that matters.
Re:Ask HN: Is there a great open source crawler? by haihai on Thursday June 03, 2010 @12:57PM (#1401818)
I considered 80legs. The problem is that I don't want to do this for just one site. I may have dozens. I can run this off my own servers/bandwidth (which I already pay for) for much cheaper than using 80legs.

If 80legs was purely usage based and cheaper I probably would have used it.

Re:Ask HN: Is there a great open source crawler? by jdrock on Thursday June 03, 2010 @01:39PM (#1401932)
Understood. We do work with folks on custom per-use plans, but we just need to figure out what your requirements are so that we can see if there's some way for us to help.