Re:Ask HN: How are you dealing with scraping hits from EC2 m...
by lifeguard on Wednesday December 12, 2012 @09:28AM
(#4910160)
That is the way to block spiders that obey the standards and enough do not that robots.txt is not a solution.
[ Parent
| on HN ]
There are no comments on this comment yet.