News for hackers. Stuff that matters.

A comment on Google's robots.txt. Parent comment

Re:Google's robots.txt by coolj on Sunday February 23, 2014 @04:31AM (#7285393)
curl -s https://raw.github.com/bluedragonz/bad-bot-blocker/master/.h... | awk -F\" '/SetEnvIfNoCase/ {pattern = pattern $2 "|"} END {print "if ($http_user_agent ~ (" substr(pattern, 0, length(pattern)-1) ")) { return 403; }"}'

Not tested. :)

[ Parent | on HN ]

There are no comments on this comment yet.