News for hackers. Stuff that matters.

A comment on AI bots everywhere. Does anyone have a good whitelist for robots.txt?. Parent comment

Re:AI bots everywhere. Does anyone have a good whitelist for... by zaphodias on Wednesday January 29, 2025 @10:02AM (#42865652)
> is a right

I believe they're saying that cloudflare will block them just for using a blacklisted client, even if they're legit users and not bots

[ Parent | on HN ]
Re:AI bots everywhere. Does anyone have a good whitelist for... by userbinator on Friday January 31, 2025 @12:48AM (#42885010)
Not even that; it's a whitelist, not a blacklist, and the only clients whitelisted are essentially those of Big Browser.
[ Parent | on HN ]