News for hackers. Stuff that matters.
Re:Google's robots.txt by bmelton on Sunday February 23, 2014 @03:13AM (#7285305)
That's amazing. Know of anything like it for Nginx?
Re:Google's robots.txt by anon4 on Sunday February 23, 2014 @04:04AM (#7285358)
It's blocking by user agent and source ip. You should be able to port the list easily for Nginx. I'd even say you can write a simple awk script in a few minutes to convert from Apache's format to Nginx's.
Re:Google's robots.txt by coolj on Sunday February 23, 2014 @04:31AM (#7285393)
curl -s https://raw.github.com/bluedragonz/bad-bot-blocker/master/.h... | awk -F\" '/SetEnvIfNoCase/ {pattern = pattern $2 "|"} END {print "if ($http_user_agent ~ (" substr(pattern, 0, length(pattern)-1) ")) { return 403; }"}'

Not tested. :)