It's just for us oldies who prefer the ethical web where people respected the RFCs they have collectively written.

What will be the consequence? My risky prediction: Absolutely Nothing, for neither goggle nor any of the other miscreants running the bot nets.

And just to take a poke at the title of the original El Reg article: Do you really think the problem is that they just haven't "learned" yet how to "play nice"? I'm sure the great oracle goggle, just needs someone to poperly splain it to them.

Probably need to collect logs across multiple sites though, as some crawlers would exclude your domain once they knew.

2025-03-05 13:32:09 45.89.148.57 (AS46844 SHARKTECH) "Mozilla/5.0 (Windows; U; Windows NT 6.1; pt-BR; rv:1.9.2.18) Gecko/20110614 Firefox/3.6.18 (.NET CLR 3.5.30729)"

  $db=...
  $sth=$db->prepare("INSERT INTO beacon (time, addr, query, agent, referer) VALUES (FROM_UNIXTIME(?), ?, ?, ?, ?)");
  $sth->execute([
    $_SERVER["REQUEST_TIME"],
    $_SERVER["REMOTE_ADDR"],
    $_SERVER["QUERY_STRING"],
    isset($_SERVER["HTTP_USER_AGENT"])?$_SERVER["HTTP_USER_AGENT"]:NULL,
    isset($_SERVER["HTTP_REFERER"])?$_SERVER["HTTP_REFERER"]:NULL
  ]);