News for hackers. Stuff that matters.
Re:IndexTank / 80legs Crawlathon (developer contest) by revorad on Wednesday June 15, 2011 @05:16PM (#2659053)
Oh I didn't know about the robots.txt rule for /results. Thanks for pointing it out or I would have built my own crawler and got banned. I think I'll go with playlists.

There are no comments on this comment yet.