Re:Robots.txt is a suicide note
by G_Morgan on Tuesday May 10, 2011 @06:45AM
(#2532016)
Yeah what this really needs is a bot block filter service where bots that do not conform to expectations can be bulk banned. It is too much effort to do this manually but a larger service can work.
[ Parent
| on HN ]
There are no comments on this comment yet.