News for hackers. Stuff that matters.

A comment on Show HN: Stop AI scrapers from hammering your self-hosted blog. Parent comment

Re:Show HN: Stop AI scrapers from hammering your self-hosted... by misterchocolat on Thursday December 18, 2025 @05:50AM (#46311134)
hey! thanks for that read suggestion that's indeed a pretty funny captcha strat. Yup the links show up if you use the Lynx web browser. As for AI scrapers impersonating googlebot I feel like yes they'd definitely start doing that, unless the risk of getting sued by google is too high? If google could even sue them for doing that?

Not an internet litigation expert but seems like it could be debatable

[ Parent | on HN ]
Re:Show HN: Stop AI scrapers from hammering your self-hosted... by n1xis10t on Thursday December 18, 2025 @02:53PM (#46317732)
Yeah I guess I don’t know if you can sue someone for using your headers, would be interesting to see how that goes.
[ Parent | on HN ]
Re:Show HN: Stop AI scrapers from hammering your self-hosted... by throawayonthe on Thursday December 18, 2025 @05:08PM (#46319488)
i think making the case of "you are acting (sending web requests) while knowingly identifying as another legal entity (and criminally/libelously/etc)" shouldn't be toooo hard
[ Parent | on HN ]
Re:Show HN: Stop AI scrapers from hammering your self-hosted... by n1xis10t on Thursday December 18, 2025 @05:13PM (#46319543)
Seems like, but there are tons of things that forge request headers all the time, and I don’t think I’ve heard of anyone getting in legal trouble for it. Now I think most of these are scrapers pretending to be browsers, so it might be different I don’t know.
[ Parent | on HN ]
Re:Show HN: Stop AI scrapers from hammering your self-hosted... by owl57 on Thursday December 18, 2025 @09:21PM (#46321633)
And most of them are pretending to be Chrome. If Google had a good case against someone reusing their user agent, maybe they would already have sued?

Or maybe not. Got some random bot from my server logs. Yeah, it's pretending to be Chrome, but more exactly:

"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"

I guess Google might be not eager to open this can of worms.

[ Parent | on HN ]
Re:Show HN: Stop AI scrapers from hammering your self-hosted... by kuylar on Thursday December 18, 2025 @04:29PM (#46319052)
> As for AI scrapers impersonating googlebot I feel like yes they'd definitely start doing that, unless the risk of getting sued by google is too high?

Google releases the Googlebot IP ranges[0], so you can makes sure that it's the real Googlebot and not just someone else pretending to be one.

[0] https://developers.google.com/crawling/docs/crawlers-fetcher...

[ Parent | on HN ]
Re:Show HN: Stop AI scrapers from hammering your self-hosted... by n1xis10t on Thursday December 18, 2025 @04:33PM (#46319090)
Oh good idea!
[ Parent | on HN ]