News for hackers. Stuff that matters.
Re:Robots.txt meant for search engines don’t work well for w... by davb on Friday April 21, 2017 @03:34AM (#14163710)
Additionally, if excessive scraping became an issue for my site I'd consider rate limiting client.

There are no comments on this comment yet.