News for hackers. Stuff that matters.
Re:Only Google is really allowed to crawl the web by jedberg on Friday March 26, 2021 @06:33PM (#26597819)
> Don't do this if you actually rely on search engine traffic though. Google may get pissed and send you lots of angry mail like "There's a problem with your site".

Ah, but of course you would exclude Google's published crawler IPs from this restriction, because that is exactly what they want you to do.

There are no comments on this comment yet.