I'm on AWS and use their WAF service to do some rudimentary bot blocking, but CDNs (their Cloudflare equivalent, Cloudfront) have been too expensive in the past and the bot control mechanisms they offer have too many false positives. Perhaps I should check it out again.

Part of the problem is the economics of it -- I've chosen to self-fund a high traffic site without ads, and that's on me. But it was possible to do this just a few years ago.

>and the bot control mechanisms they offer have too many false positives. Perhaps I should check it out again.

Cloudflare no longer does CAPTCHAs so even if users get flagged as bots, the user experience isn't terrible. You just have to click on a box and you're on your way. It adds maybe 3s of delay, far better than anti-bot solutions that require you to solve an captcha, or imperva's (?) challenge that requires you to hold a button for 5-10s seconds.

If you're given a button to click, your browser has successfully passed the environment integrity checks and you have not been flagged as a bot.

You'll be flagged as a bot if your browser configuration has something "weird" (e.g. webrtc is disabled to reduce your attack surface) and you will be completely unable to access any site behind cloudflare with the anti-bot options turned on. You'll get an infinite redirect loop, not a button to click.

Note that Google's version of this was determined to be checking whether you had a 9-day-old tracking cookie.

The researcher who discovered this was able to generate 60,000 "I am not a bot" cookies per day, and use them up about 15 times each in a bot before it started getting captchas.

That was in 2016 though.

Couldn't the content of that cookie be used to validate its actual age? Like, just signing the date of generation?

...or generate a bunch and wait 9 days?

That's probably what it was. So they accessed some page over and over, pretending to not have the cookie yet, got a bunch of cookies, and 9 days later, used them to bypass captchas.

That is helpful, thank you! I will check it out.

Cloudflare is free. Give it a try.

I'm not sure why this is down voted. The forum I run was hitting monthly bandwidth limits within days because of bots. Changes to htaccess and checking environment data in the PHP code was a cat and mouse game. The free Cloudflare tier solved my problem.