I think the most likely long-term solution is something like DIDs.
https://en.wikipedia.org/wiki/Decentralized_identifier
A small number of trusted authorities (e.g. governments) issue IDs. Users can identify themselves to third-parties without disclosing their real-world identity to the third-party and without disclosing their interaction with the third-party to the issuing body.
The key part of this is that the identity is persistent. A website might not know who you are, but they know when it’s you returning. So if you get banned, you can’t just register a new account to evade the ban. You’d need to do the equivalent of getting a new passport from your government.
Are you sure you read my comment fully?
the governments powerful enough to roll something like this out are not trusted authorities which will protect the privacy of their citizens. remember before the Snowden revelations when the NSA's director of national intelligence swore under oath that they did not collect "any type of data at all on millions of Americans"?
https://en.wikipedia.org/wiki/James_Clapper#Testimony_to_Con...
Online identity verification is probably best handled by an organization with that as a single priority.
Under the government ID scheme, we have to trust [bad corrupt government] to verify all citizens of [bad corrupt government]. Since that government frequently lies and acts maliciously using every means at their disposal, platforms will treat IDs verified by that government similar to bot traffic and the country will be cut off from the public internet. You'll be banning scientists and journalists from working with others around the world, just because they live in a country with an obnoxious government.
Isn't it also best if people can have multiple identities? Or should someone's contributions to X field be discounted because of their dabbling in fringe Y field?
The trust I mentioned was the ability for third-parties to trust that the authority will not hand out IDs in an uncontrolled manner. I was not saying that the ID holders need to trust the authority:
> Users can identify themselves to third-parties without disclosing their real-world identity to the third-party and without disclosing their interaction with the third-party to the issuing body.
If the authority doesn’t know how your ID is used, you don’t have to trust the authority to keep that information private.
If the only way to associate a user with their ID is by fingerprinting them, you can do the same thing without an ID with having shadow profiles. If the proof system is designed for privacy, the ID doesn't make you more trackable.
In other words, if the ID never directly leaks companies can just make up a static ID for you and get the same results.
There is a permanent ID, but it doesn't have to be told to the site.
In which case it doesn't make tracking any easier than the site making up a "fake" ID for you.
This doesn’t make sense. The whole point of using IDs in this way is in an authenticated context.
Did you think I was suggesting that this ID would be accessible to any website without asking? This is something you would send as part of a registration step. So, for instance, if you spam Hacker News, you get banned, you try to register again, it receives the same ID as before and knows not to let you register.
As you note, websites can already force people to register, so this isn’t adding anything new there.
This concept has been studied already extensively, e.g [1] (in 2000!) by people like Rivest and Chaum, who have actual decade-old competence in that field.
[1] https://people.csail.mit.edu/rivest/pubs/pubs/LRSW99.pdf
https://www.nytimes.com/2006/02/05/technology/postage-is-due...
If you think people self-censoring themselves on social media is now a problem (the "unlive" novlang is always such a dystopic hint to me), you have seen nothing.
The whole point of having trusted issuers is that there aren’t any “disposable companies” who hand out many identities in an uncontrolled manner. If there were, they would quickly become untrusted, making the IDs worthless.
> I should be able to control people on the internet better
This is so twisted out of all proportion I find it difficult to believe you are being genuine.
A government giving an ID to somebody is not propping up “an untenable business model”.
Saying “I don’t want you to access my service” is not “controlling people on the Internet”.
E.G: youtube has a disproportionate share of the attention market, therefore can control what people think, and they are heavily influenced by big companies that will tell users what they can say or not on the platforms.
Your account on youtube is linked to your phone, emails, photos, calendar... So do something that one of those companies don't want you to do because it's not profitable for them,
And of course, they change the rules of the time.
That's why I don't have a google account. But most people are not capable of such insights, they don't have the knowledge to realize how much they can be screwed.
What happened to this being a case of the government propping up an untenable business model?
You don’t have a solid view point, you’re just saying whatever springs to mind that disagrees with what I say. This isn’t a genuine conversation, you’re just arguing for entertainment. I’m out.
Hell. No.
The ultimate challenge is to replicate end-to-end natural human cognition, which is currently an unsolved and hard problem (and also not necessarily the main focus of AI researchers).
Are you sure? And how do you know?
There are a lot of CAPTCHA cracking services. Given the price, they are hardly sustainable even under developing country wage level. I believe they actually solve the easy ones automatically and humans are only involved for the harder ones.
This is wrong, badly wrong.
CAPTCHA stood for “Completely Automated Public Turing test to tell Computers and Humans Apart”. And that’s how people are using such things: to tell computers and humans apart. But that’s not the right problem.
Spam and abuse can come from computers, or from humans.
Productive use can come from humans, or from computers.
Abuse prevention should not be about distinguishing computers and humans: it should be about the actual usage behaviour.
CAPTCHAs are fundamentally solving the wrong problem. Twenty years ago, they were a tolerable proxy for the right problem: imperfect, but generally good enough. But they have become a worse proxy over time.
Also, “human-friendly CAPTCHAs” are just flat-out impossible in the long term. As you identify, it’s only a “for now” thing. Once it’s a target, it ceases to be effective. And the range in humans is so broad that it’s generally distressingly easy to make a bot exceed the lower reaches of human performance.
> Proof-of-work CAPTCHAs work well for making bots expensive to run at scale, but they still rely on accurate bot detection. Avoiding both false positives and negatives is crucial, yet all existing approaches are not reliable enough.
Proof-of-work is even more obviously a temporary solution, security by obscurity: it relies upon symmetry in computation power, which is just wildly incorrect. And all of the implementations I know of have made the bone-headed decision to start with SHA-256 hashing, which amplifies this asymmetry to ludicrous degree (factors of tens of thousands with common hardware, to tens of millions with Bitcoin mining hardware). At that point, forget choosing different iteration counts based on bot detection, it doesn’t even matter.
—⁂—
The inconvenient truth is: there is no Final Ultimate Solution to the Spam Problem (FUSSP).
> Productive use can come from humans, or from computers.
I agree in principle, but the reality is that 37% of all internet traffic originates from bots[1]. The overwhelming majority of that traffic (89% according to Fastly) can be described as abusive. In turn, the abusive traffic from humans likely pales in comparison. It's vastly cheaper to setup bot farms than mechanical turk farms, and it's only getting cheaper.
Identifying the source of the traffic, while difficult, is a generalizable problem. Whereas tracking specific behavior will depend on each site, and will likely require custom implementation for each type of service. Or it requires invasive tracking of users throughout the duration of their session, as many fraud prevention systems do.
Both approaches can be deployed at the same time. A CAPTCHA is not meant to be the only security solution anyway, but as a first layer of defense that is generally simple to deploy and maintain.
That said, I concede that the sentence "[CAPTCHAs] are the only solution" is wrong. :)
> Proof-of-work is even more obviously a temporary solution, security by obscurity
I disagree, and don't see how it's security by obscurity. It's simply a method of increasing the access cost for abusive traffic. The more signals are gathered that identify the user as abusive, the higher the "price" they're required to pay to access the service. Whether the user is a suspected bot or not could just be one type of signal. Behavioral and cognitive signals as mentioned in TFA can be others. Yes, these methods aren't perfect, and can mistakenly penalize human users and be spoofed by bots, but it's the best we currently have. This is what I'd like to see improved.
Still, even with all their faults, I think PoW CAPTCHAs offer a much better UX than traditional CAPTCHAs ever did. Yes, telling humans apart from computers is getting more difficult, but it doesn't mean that the task is pointless.
[1]: https://learn.fastly.com/rs/025-XKO-469/images/Fastly-Threat...
It takes a long time and enormous amounts of money to make new chips for a specific proof of work. And sites can change their algorithm on a dime. I don't think this is a big issue.
It’s important to assume, in security and security-adjacent things, that the attacker has more compute power than the defender. You cannot win in this way.
Proof-of-work is bad rate limiting that relies upon the server having a good estimate of the capabilities of the client. No more, no less.
I bring up the SHA-256 thing as an argument that none of the players in the space are competent. None of them. If you exclude hand-rolled cryptography or known-bad techniques like MD5, SHA-256 is very literally the worst choice remaining: its use in Bitcoin and the rewards available have utterly broken it for this application. If you intend proof of work to actually be the line of defence, you start with something like Argon2d instead. I honestly think that, at this stage, these scripts could replace their proof of work with a “sleep for one second” (maybe adding “or two if I think you’re probably a bot”) routine and have the server trust that they had done so, without compromising their effectiveness.
“Expensive” depends on the value of what you do behind the captcha
There are human-solving captcha services that charge USD 1 for 1k captchas solved (0.1 cents per captcha)
So as long as you can charge more than what solving the captchas cost, you are good to go
Unfortunately, for a lot of tasks, humans are currently cheaper than AI
https://en.bitcoin.it/wiki/Hashcash
https://en.wikipedia.org/wiki/Hashcash
C implementation (feature-rich): https://github.com/hashcash-org/hashcash/tree/master/c
A Factor (Forth-like language) implementation of it: https://github.com/factor/factor/blob/master/extra/hashcash/...
2. Send link to coworkers via Slack so they can spend five minutes doing the tasks.
3. Capture that data and create thousands of slight variations saved to db as profiles
4. Bypass bot protections.
There is nothing anyone can do to prevent bots.
Are you sure about this?
At the time, Ticketmaster’s anti-bot measures were the gold standard. They gave us fair warning that they planned to implement Mastercard’s SaaS-based solution (same as described in OP’s article), so I had everyone on the team capture keyboard-typing cadence, mouse movements, and other behavioral metrics. I used that as the excuse to build a Chrome extension that handled all of those tasks, and I leaned on the backend team to stop procrastinating and integrate the new API endpoints that Ticketmaster was rolling out. For about a week, that extension managed millions of dollars in inventory—until I got our headless browsers back up and running.
In the end, any lock can be picked given enough time; its only real purpose is to add friction until attackers move on to an easier target. But frankly, nobody can stop me from scraping data or automating site interactions if it’s more profitable than whatever else I could be working on. I have some ideas how to prevent me from using automated bots but all of the companies I've applied to over the years never respond -- that's on them.
No they don't, that's the point: you can serve everyone a PoW and don't have to discriminate and ban real people. This system you're enthusiastic about is what tries to do this "accurate bot detection" (scratch the first word)
https://github.com/TecharoHQ/anubis/blob/main/data/botPolici...
Distinguishing en mass seems like a waste to me. Deal with the actual problems like resource abuse.
I think part of the issue is that a lot of people are lying to themselves that they "love the public" when in reality they really don't and want nothing to do with them. They lack the introspection to untangle that though and express themselves with different technical solutions.
I work for Stytch and for us, that looks like:
1) make it easy to provide Connected Apps experiences, like OAuth-style consent screens "Do you want to grant MyAgent access to your Google Drive files?"
2) make it easy to detect all bots and shift them towards the happy path. For example, "Looks like you're scraping my website for AI training. If you want to see the content easily, just grab it all at /LLMs.txt instead."
As other comments mention, bot traffic is overwhelmingly malicious. Being able to cheaply distinguish bots and add friction makes your life as a defending team much easier.
They're the only solution that doesn't require a pre-existing trust relationship, but the web is more of a dark forest every day and captchas cannot save us from that. Eventually we're going to have to buckle down and maintain a web of trust.
If you notice abuse, you see which common node caused you to trust the abusers, and you revoke trust in that node (and, transitively, everything that it previously caused you to trust).
The problem is that such a system could be easily abused or misused. A bad actor could intentionally or mistakenly penalize users, which would have global consequences for those users. So we need a web of trust for the judges as well, and some way of disputing and correcting the mistake.
It would be interesting to prototype it, though, and see how it could work at scale.
I don't see this ever happening, though.
I don't think there should be any judges (or to put it differently, I think every user should be a judge), nor any centralized database, no roots of trust at all. That way it doesn't present any high value targets for corruption to focus on.
The trustworthiness of a user in some domain (won't-DOS-your-page could be a trust domain, writes-honest-product-reviews could be a domain, not-a-scammer, etc) as evaluated by some other individual would have to do with some aggregation of the shortest paths (and their associated trust scores) between those to users on the trust graph.
There is no trust score for user foo, only a trust score for user foo according to user bar. User baz might see foo differently.
If you get scammed, you don't revoke trust in the scammer. Well, you do, but you also go one-hop-back and revoke trust in whoever caused you to trust the scammer. This creates incentives towards trust hygiene. If you don't want people to stop trusting you, then you have to be careful about who you trust. It's a protocol-level proxy for a skill we've been honing for millenia: looking out for each other.
But it doesn't work if there's just a single company that tracks your FICO score or something like that. Either that company ends up being too juicy of a target and ends up itself becoming corrupt, or people attack the weak association between user and company such that the company can't actually tell the difference between a scammer and a legit user (the latter is the case for the credit score companies, hence: identity fraud).
Attacks like that are much harder to pull off if the source of truth isn't some remote database somewhere and is instead based on the set of people you see every day in meatspace.
https://docs.zkpassport.id/faq https://docs.zkpassport.id/examples/personhood
Y'all will balk at that but in a decade or so I think we'll have no other choice.
But even that will fail since certain countries will likely be less precious about their system for this and spammers will still get fake ids. Same problem as now with phone numbers/rcs spam.
Proof-of-work CAPTCHAs work well for making bots expensive to run at scale, but they still rely on accurate bot detection. Avoiding both false positives and negatives is crucial, yet all existing approaches are not reliable enough.
One comment re:
> While AI agents can theoretically simulate these patterns, the effort likely outweighs other alternatives.
For now. Behavioral and cognitive signals seem to work against the current generation of bots, but will likely also be defeated as AI tools become cheaper and more accessible. It's only a matter of time until attackers can train a model on real human input, and inference to be cheap enough. Or just for the benefit of using a bot on a specific target to outweigh the costs.
So I think we will need a different detection mechanism. Maybe something from the real world, some type of ID, or even micropayments. I'm not sure, but it's clear that bot detection is at the opposite, and currently losing, side of the AI race.