News for hackers. Stuff that matters.
Re:Cloudflare’s CAPTCHA replacement with FIDO2/WebAuthn is a... by oefrha on Friday May 14, 2021 @09:33AM (#27154111)
I think the problem is some IPs just straight-up always get CAPTCHAs from Cloudflare even if one’s a good netizen, respect robots.txt, not crawl at ridiculous rate, and not lie in the user agent. One reason is shared IP, which disproportionally affects people from third world countries as their ISPs don’t have enough IPv4 for everyone; but it also happened mysteriously to at least one dedicated IP I used in the past. Your confrontational tone is rather unfortunate, and the problem of course is that you don’t guarantee anything even if the user has done nothing wrong, as is manifest from the choice of the phrase “starting point”.
Re:Cloudflare’s CAPTCHA replacement with FIDO2/WebAuthn is a... by ehutch79 on Friday May 14, 2021 @09:57AM (#27154355)
Then the problem has nothing to do with your crawling.