News for hackers. Stuff that matters.
Re:What one may find in robots.txt by protomyth on Monday May 18, 2015 @11:07AM (#9564861)
This is silly. If you have a user of your site going through your bots file and specifically going to directories listed as Disallow then you deal with that user. Blocking based on the robots.txt for a directory that doesn't exist anywhere but that file is fine. I did a two bad directory ban, it seemed to work fine.

There are no comments on this comment yet.