News for hackers. Stuff that matters.
Re:Who exactly is crawling my site? by mrgreenfur on Tuesday October 22, 2013 @11:57AM (#6592756)
I'm late, but for this there are companies that sit in the middle and block crawlers, spambots and known-malicious addresses. Here's one I use: http://www.incapsula.com/ So far I've been happy with it!

There are no comments on this comment yet.