Re:AI crawlers haven't learned to play nice with websites
by acdw on Tuesday March 18, 2025 @10:13AM
(#43399668)
could you share the code for that? thinking I could set up a fail2ban rule for these ips
[ Parent
| on HN ]
Re:AI crawlers haven't learned to play nice with websites
by blueflow on Tuesday March 18, 2025 @10:18AM
(#43399739)
its a pathetic piece of php code:
$db=...
$sth=$db->prepare("INSERT INTO beacon (time, addr, query, agent, referer) VALUES (FROM_UNIXTIME(?), ?, ?, ?, ?)");
$sth->execute([
$_SERVER["REQUEST_TIME"],
$_SERVER["REMOTE_ADDR"],
$_SERVER["QUERY_STRING"],
isset($_SERVER["HTTP_USER_AGENT"])?$_SERVER["HTTP_USER_AGENT"]:NULL,
isset($_SERVER["HTTP_REFERER"])?$_SERVER["HTTP_REFERER"]:NULL
]);