News for hackers. Stuff that matters.
Re:Google's robots.txt by anon4 on Sunday February 23, 2014 @04:04AM (#7285358)
It's blocking by user agent and source ip. You should be able to port the list easily for Nginx. I'd even say you can write a simple awk script in a few minutes to convert from Apache's format to Nginx's.

There are no comments on this comment yet.