Re:HTTP 402 – Payment Required
by sbierwagen on Friday June 06, 2014 @11:44AM
(#7858559)
On bbot.org I return 410 on robots.txt, for tedious ideological reasons. Doesn't stop spiders requesting it, of course, since that check is probably hard-coded.
[ Parent
| on HN ]
There are no comments on this comment yet.