News for hackers. Stuff that matters.

A comment on Show HN: Stop AI scrapers from hammering your self-hosted blog.

Re:Show HN: Stop AI scrapers from hammering your self-hosted... by jakub_g on Friday December 19, 2025 @08:49AM (#46325828)
> checks user agents and won't show the links to legitimate search engines, so Google and Bing won't see them.

Serving different contents to search engines is called "cloaking" and can get you banned from their indexes.

[ Parent | on HN ]
Re:Show HN: Stop AI scrapers from hammering your self-hosted... by andersmurphy on Friday December 19, 2025 @11:21AM (#46327488)
Somehow doubt this. It would mean most react websites that serve static content without paywalls for SEO would get banned by the indexes too.

Which for better or worse is a large portion of the modern internet.

[ Parent | on HN ]
Re:Show HN: Stop AI scrapers from hammering your self-hosted... by misterchocolat on Friday December 19, 2025 @08:53AM (#46325861)
didn't know that thanks for pointing it out, i'll remove that feature
[ Parent | on HN ]