News for hackers. Stuff that matters.
Re:What one may find in robots.txt by cmdrfred on Monday May 18, 2015 @03:19PM (#9566316)
I'm with you on this, not for huge sites but if even 1% of sites used this behavior malicious bots would be forced to honor robots.txt

There are no comments on this comment yet.