News for hackers. Stuff that matters.
Re:Ask HN: How are you dealing with scraping hits from EC2 m... by lifeguard on Wednesday December 12, 2012 @09:28AM (#4910160)
That is the way to block spiders that obey the standards and enough do not that robots.txt is not a solution.

There are no comments on this comment yet.