Re:Robots.txt meant for search engines don’t work well for w...
by davb on Friday April 21, 2017 @03:34AM
(#14163710)
Additionally, if excessive scraping became an issue for my site I'd consider rate limiting client.
[ Parent
| on HN ]
There are no comments on this comment yet.