Re:Perplexity is using stealth, undeclared crawlers to evade...
by 1vuio0pswjnm7 on Tuesday August 05, 2025 @01:06PM
(#44800820)
(IMHO) The correct way to limit "abuse", e.g., by "bots", is to rate limit. But as other commenters point out, Cloudflare routinely (and knowingly) blocks humans sending only a single GET request, e.g., with Javascript disabled. Needless to say, this does not exceed any reasonable rate limit. It is not "abuse". By Cloudflare's own admission, and as demonstrated by the case of Perplexity AI, this "bot protection" does not stop "bots".
It does stop any humans not using popular advertising-sponsored web browsers.
It does stop any humans not using popular advertising-sponsored web browsers.