On a somewhat related note: I used to write sneaker buying bots and captcha was one of the best things to happen to the bot industry.
Captcha was easily "bypassed" by services that had humans sitting at computers generating tokens. Recaptcha tokens are valid for two minutes by default, so you'd be able to generate tokens up to two minutes in advance and have dozens of tokens available when the product in question became available for purchase. Real buyers had to wait for drop and then spend 15s+ filling out the captcha. Bots would take most of the stock in less than a second. I always felt like this was a fantastic example of a bot prevention mechanism that actually actively harmed "legitimate" users.
Can I ask why you quit the Bot industry? It seems super lucrative right now.
You could actually call this a symbiotic lock-in. I once read an excellent post on how SEO (despite being a gray area for search engines) entrenches Google's market share because many SEO practitioners specialize in it.
If your ecosystem has become so specialized that most well-adapted users are bots, you have a problem (or not, if you can monetize bots).
This is the whole point of this article: device detection is generally a false sense of security.
It really depends on what the benefits of bypassing the detection are and if those benefits can be had elsewhere at lower cost. For ticket sales or limited run product sales or similar things, the benefits of buying (and presumbably reselling) the limited item is high, and there's a limit to what you can do to detect humans (and as you've described it can be counter productive). For spam prevention, making it cost more to spam means spammers are encouraged to find somewhere else to spam, which is good for your users (while using your product anyway). But it won't stop everything, some people are going to manually type their spams, and some people will build robots to tap out their spams more like a person; and if you ruin the experience for users (especially new users) that doesn't work either; a network with zero messages is spam free, but doesn't help anyone.
Not sure how reCaptcha “v3” (the photos of streets one) works. It could work the same way, but just with more known and unknown (unclassified) pictures? It’s also a lot more harsh if you get it wrong: artificial delays, switching to the “click until no more remain” mode (with more delays), and more. Sometimes, it just refuses to let you through no matter what: https://youtu.be/zGW7TRtcDeQ It’s extremely frustrating.
My own experience briefly running sites - bot traffic is 100% trash on almost every metric. Actual conversions, spam, click fraud, other fraud (spam talking about work from home), privacy violations (trying to scrap all user profiles, capturing deleted user content etc).
The real reasons people implement these draconian measures range from inept cargo culting to nefarious business models. If you have a problem with spam, add user moderation and call it a day. That’s just a justification though, there is never a pro-user reason to destroy ux and privacy.
Huh?
I require users to register (little to no privacy) and pay a bit - this results in a both better user experience and is a pro-user reason to destroy privacy. You can also blacklist most of the anon email providers / do SMS verification etc
I’m also curious what your site is. Will you say, or do you wish to remain anonymous?
site owners want the advertising that was embedded to be seen and thus make revenue from it.
Slack limits the amount of searchable messages to 10k, so if you save them regularly to local DB, you can search without limits.
All kinds of owners don't sell/show ads, or care about scraping.
There is no need to destroy privacy, waste resources, create terrible ux, gate content…
Automation detection and privacy are orthogonal. The ideal bot detector yields a single bit of information: bot or not. It doesn't need to try to de-anonymize users or anything like that. As for UX, well, spam is a bad UX.
The incentive is money. You can't just wave a magic wand and get rid of the incentive to spam things. Asserting that you can is wishful thinking.
Things that work for small sites fall apart once you get 10~100 million+ people.
Look, I help do the actual user moderation for a large (10m+) subreddit, and it is a god damn nightmare. Plus, on top of that, we have to do automated bot detection anyways, because otherwise it's just neverending bot spam.
Reddit is actually a good example of having millions of users with no captcha. They “fixed” their problem with user moderation.
You chose to become a moderator, so it must not be that much of a nightmare.
EDIT: Reddit does have a captcha now. My guess is it came along with the other users hostile changes in the new design.
edit: They have for at least the last 6 years based on online questions. Long before the new design.
That's why most of them seem to be perpetually banned.
[edit] Now, if someone would build an onion router that ran on people's end devices and served as a proxy network for these front ends, that would be interesting ... but would probably end up getting lots of people blocked, so please don't take it seriously.
These scum companies rely on "engagement", so they can only block people if it's a minority. If _everyone_ participates, they have no choice but to accept it (or maybe change their product so there's no incentive to run these unofficial front-ends in the first place).
https://www.bleepingcomputer.com/news/security/microsoft-adm...
How else would you test your SPA against millions of different content-modifying browser plugins, for example.
You don't.
Ideally, you could get rid of every captcha and just have an open API for everything. Somebody wants to create lots of email addresses to send spam? Remove the incentive to send spam.
Somebody posts thousands of manipulative tweets? Maybe the "feed" model of social media is wrong. Don't assume every random account is equally important. Better just have a simple social network where you can connect with your friends. Or even better, design your society such that it's decisionmaking is more resilient and can't be gamed.
The really hard but worthwile problem is not in tech, it is how we evolve our society along with the tech.
Unfortunately a new generation has come along that believes advertising-supported business models are their birthright.
I respect that services have running costs and I’m happy to cover my usage, but I have to understand that the average consumer doesn’t give a crap, or is simply ignorant and doesn’t know.
Good luck. Literally nobody has figured out a viable way to do that. At this point, we need to consider spam a fact of nature, like viruses.
> Locking down things takes the fun out of them.
Locking down things so we create a space that actually functions and doesn't succumb to abuse. What you call fun, others would call trolling they'd like to get rid of.
But how?
What if you only pop up important information via an onBlur event in a dynamically created div element that is overlayed on the page, only after asynchronously querying a REST endpoint to get information from a database?
How would a crawler even know that this element is shown when you leave a specific input field? How would it know to wait a second while the database is getting the data?
Like it or not, we're well past the days of complete HTML documents being served for a request.
I find it's very useful to think of the problem not as governed by technical possibility, but rather by costs.
Good spoofing detection does not prevent fraud, it increases its costs to fraudsters (hoping to render fraud uneconomical).
Bad spoofing detection harms legitimate users and is inefficient against capable actors.
All of this gets much more interesting and complex when we consider privacy (and privacy compliance) to be part of the puzzle - now fraud prevention incurs a cost to the defender as well.
Finally, it's important to consider asymmetric risk/benefit counts. Are you defending against a single major heist, or against billions of tiny events (click fraud)? The trade-off will look different.