Do CAPTCHAs in practice really serve to perfectly tell apart humans or robots, or just to thwart the laziest attempt of abusing a website? Everyone knows that a door lock, no matter how good, can be overcome if you a really determined. But the lock still thwarts off opportunists. Yes, the goal when we first developed CAPTCHAs may have been to tell apart humans and robots. Realistically now, if CAPTCHAs can still significantly reduce the traffic from bots then companies will keep using them.

The difference with a door lock is, even if you are sufficiently determined and have a way of defeating particular kinds of door lock with 95% confidence, that doesn't mean that you can instantly break into 95% of houses that use that lock, because you are one person with a physical body.

If bots that can break CAPTCHAs become widespread, the volume of spam, scams, and other junk traffic is going to cause problems for many people and small websites.

No, because bots that handle spam get better as well:

https://thespamchronicles.stavros.io/welcome/

For a long time now CAPTCHAs only make bots more expensive. There are plenty of (human-staffed) services that will solve your captchas for $0.001-$0.005 per solved captcha. Better AI lowers the cost, but it's not necessarily a massive change of the status quo.

In a way that is silly:

Increase the cost of having bots do it, while...

...also increasing the time wasted by humans. Time which is considered much more costly/valuable than whatever $ value spent on the bots.

So maybe the time has come to regard captcha's pointless, and just drop that nonsense.

That was true of early captchas but Google changed it from 'wasting humans time' to 'using humans time to train their machine learning algorirhms'

about to say the same - but to be clear, it is "training computer vision models" a specific use of machine learning

The difference between zero and nearly zero can be pretty large at scale.

I would legitimately pay for this. I hate doing captchas and the internet seems to hate Firefox.

Not necessarily, if the messages are analyzed there's usually an intent (solicitation, scam, etc.) so you can filter based on the content similar to email. The shorter lead forms without a message to analyze are a bit different since you can only go based on IP, email, phone but it's possible.

Solving the puzzle is only of the requirement to getting past captchas. For better or worse most captchas use IP(subnet) based rate limiting and cross site tracking as well, so the physical body argument holds in captchas as well. Try using the web inside free VPN or Tor and see if you can break captchas.

CAPTCHA is about more than spam.

Big Tech built their empires on scraping and stealing data (How do you think LinkedIn or Facebook got started?) But when we try to scrape them they throw a massive hissy fit, and so they put a lot of engineering into CAPTCHA systems to keep their data locked away. Eventually, the pinnacle of bots will be something that reads the framebuffer, and manipulates a mouse and keyboard to scrape websites.

This has got these companies freaked out, because all the founders know the dirty secret of their origin story, and that someone else can come along and do exactly the same thing to them.

(This, of course, isn't how LinkedIn or Facebook got started)

Don't know about LinkedIn, but aren't the roots of Facebook traced back to scraping pictures of women off of Harvard's website?

Let’s just say I have insider knowledge, working for the same investors, long before the company even had 100 employees. LinkedIn was built on scraping and stolen data. Same as FB. You can deny it but it doesn’t change reality.

> or just to thwart the laziest attempt of abusing a website

Thus the XKCD CAPTCHA: https://xkcd.com/810/

I’ve seen that one before, but I got a fresh laugh out of it.

We’ll have to change the game again now that bots can beat humans at it.

I wonder if part of the solution is for us to start designing the Internet for bots. We're entering a phase where websites are coded by bots, content of all stripes (text/video/audio/images) is generated by bots, and increasingly that fact doesn't necessarily mean that we want to block it.

The main tenets of apps that are resilient to bot spam are 1) scalability, so that they can handle huge quantities of traffic and bot-generated content, and 2) the ability to differentiate high quality from low quality content (regardless of whether it was created by a human). Ironically, AI is probably the solution to the second problem.

I think what you’re proposing applies to good bots.

But the bots I wrote to get unfair advantages in auctions or tickets are probably of the kind most bots are.

To that point, when I do the picture captcha (Select the crosswalks type question), I always click a square I know is not valid and then de-select it. Adds some "human-ness" to the interaction and I never get a 2nd challenge that way. Will that be the future? Look for behavior that is too perfect?

This would be fairly simple to simulate using an AI though.

I feel like Captcha already takes into account of how quickly I select the pictures already. If I spend the time to get it perfect, I end up with more challenges than if I just select quickly based on my instincts.

If you assume that most humans spend the minimum effort possible on their captchas, your gut response is also more in line with other responses than your well thought-out response. Even a matching based on the selected squares would pick up on this.

I do the same but with erratic mouse movements

So in near future, humans will have to out-do bots in the art of make-human-looking mistakes?

Humans will lose!! lol ;-)

Sometimes I click the "I am not a robot" checkbox without even thinking. Then I panic for a moment as there's no option to uncheck.

This sounds like something a robot would say...

I had to intentionally get worse, such as not identifying the back of a street sign as a street sign.

breaking: AI is better than humans at appearing human when completing CAPTCHAs

> until we train bots to do this

This is trivial.

I always thought that's what captchas evolved into anyways. Human-reinforced training for AI image recognition.

That seems to square with the content I'm frequently tested in. Crosswalks and traffic signs for self driving cars and stairs for walking robots.

I've also noticed that captchas are getting more difficult. Is that because the AI needs to sharpen recognition skills or because that's needed for differentiating human from 'bot?

I feel like they are sometimes already doing that, I had multiple CAPTCHAs where I thought I miss clicked, but got through anyways.

I think only some parts of a CAPTCHA are challenging you (data is labeled sufficiently for mistakes to be considered a failed solve), and the other parts are still in the process of being labeled, which you are helping to accomplish, so those have no influence on the immediate outcome.

At least that's how the old "type these two words" CAPTCHAs worked. It was crowdsourced human OCR of whatever text the machine OCR couldn't make sense of. I'm not sure if "find the bus/motorcycle/crosswalk/light" is the same way, but perhaps, and it does seem to offer leeway when there's only a few pixels of that item in the frame.

What’s sad is, I’ve started to predict how dumb other humans are at filling in CAPTCHA’s. Often there’s a piece of motorcycle, stair, traffic light, fire hydrant, or non-bus picture that users will mis-select, and I’ll do what I think the masses do.

I used to choose correctly but being sent through 5 chains of CAPTCHAs is modern hell.

I'm not certain that that's what is happening, but I strongly suspect it. It is if nothing else a useful mental model: switching my strategy from selecting every square that contains foo to selecting every square that I think that Joe Schmoe would select gets me through CAPTCHAs more consistently.

It’s been like that for years, they probably use a subset for extra annotation.

> maybe we should flip the acceptance criteria to make sure you are not performing task at superhuman level

Many CAPTCHAs already operate this way. "Bots can do things at suspiciously superhuman speed" isn't new.

I said level not speed, obviously something sending solve request back less than a few milliseconds later is not human.

I have to solve all the captchas for my wife. I don’t know what that says about her, or me, but I definitely loathe them.

What it says is that one viable way to get rid of these stupid things may involve ADA lawsuits. Hopefully anti-discrimination lawyers are paying attention. Google has some pretty deep pockets, as do many websites that employ CAPTCHAS.

Failing that, we can't be more than a couple of years away from generalized solvers that can simply be implemented as browser plugins, at least on the desktop. The job of coming up with a fair, equitable and non-discriminatory test that only humans can pass is going to be an impossible one.

We do this with some "slide this puzzle piece into place" CAPCHAs; my understanding is the detection is based on how slowly you fit the piece in. A computer would be linear in its movement, whereas a human with a mouse would operate with some unevenness and/or slowness and/or inaccuracy.

Which seems easy enough to program around if you are trying to commit fraud against whatever is gated by these tools.

Get humans to do a few hundred, log their movements, sample from the distribution, sleep(t), and voilà.

One would presume it's harder than that, given it's the product of a team/company specializing in detecting exactly such a thing, right?

If you look behind the curtain in a lot of industries, many of them are held together with chewing gum and string, but have big marketing departments.

Sure, but less so on the core value prop for that company.

I feel like replaying from 100 randomly-selected but real human movements would be pretty hard to detect.

> On an unrelated note, I have found captchas that don't even work and reprompt me all the time

Modern captchas do a lot of background work regarding how human your inputs look, whether you have a human-seeming fingerprint, etc. Often when I'm behind a VPN and on Linux I have the same issue, because my setup simply looks "too botty" no matter how good I am at telling which squares have a firetruck in them.

> On an unrelated note, I have found captchas that don't even work and reprompt me all the time

This is by design. The ones where you have to identify a bus, crosswalk etc are all used to train ML models. Your results are checked against other for the captcha, but sometimes you are the first person to see the image and there’s no way to check your answer so you’ll always get served another.

Another smart thing is that they actually segment the picture by moving the squares slightly.

> This is by design. The ones where you have to identify a bus, crosswalk etc are all used to train ML models. Your results are checked against other for the captcha, but sometimes you are the first person to see the image and there’s no way to check your answer so you’ll always get served another.

Do you have a reference for this? I wouldn't have thought a process like that would be needed now-a-days for training ML models.

Not sure if there is anything directly from the horse’s mouth, but there are lots of articles describing it: https://www.techradar.com/news/captcha-if-you-can-how-youve-...

Human labels are absolutely still needed, for now at least.

Google was using CAPTCHA data to train its Street View photos for addresses and street names back in 2012*.

https://techcrunch.com/2012/03/29/google-now-using-recaptcha...

So it is actually a little different than what is noted above. I actually was told this directly from the mouth of someone who worked on this project. I don't believe it is that secret. But this is how it works.

The Captcha presents you with 9 squares. It selects a identification test at random (crosswalks, trains, buses, stoplights, etc). For this example let's say the identification test is to identify crosswalks. The squares are then filled as follows:

1) Two of the squares are requested that pass the identification test at an alpha value p < 0.05 (meaning it is more than 95% confident it IS a crosswalk).

2) One square is requested that passes the identification test at an alpha value of p < 0.01 (meaning 99%+ confident, effectively certain it IS a crosswalk)

3) One square is requested that fails the identification test at an alpha value of p < 0.01 (it is almost certainly NOT a crosswalk)

4) Two squares are requested that fail the identification test at an alpha value of p < 0.05 (it is 95% confident that it is NOT a crosswalk)

5) Three squares are requested that need have low confidence intervals p > 0.05

The captcha then shuffles these 9 images at random, it offsets the images a little bit by altering the crop slightly to prevent memorization by bots. Then it presents these 9 squares to the users asking them to identify according to the identification test.

The captcha scores the user based on their selection with the 6 known squares. The response you give on the 3 low-confidence squares has zero impact on you passing or failing the test. From what I was told, you must successfully identify both of the 99% interval squares correctly (one that passes the id test and one that doesn't). That is a hard pass/fail. From there, the captcha scores your response on the 95% confidence interval squares to the expected values. It compares that to other variables such as the speed that you answer them, the movement of the cursor and other variables (such as selecting, deselecting, etc). It also compares IP address google session data as part of its determination to determine the liklihood of humanity in the user. My understanding is that is is moderately forgiving. If the user is determined to be human based on those responses, then your responses are fed back into the confidence intervals for all of the images presented. Data by users that fail the Captcha is discarded so it doesn't feed into the confidence metrics of the images presented.

From what I was told, you can actually incorrectly identify 2 squares and still pass the captcha. The IP address and mouse movement plays a signficiant impact in the response as well as your ability to identify the two known squares.

Three of the squares are entirely unknown to the bot. You are purely feeding the confidence on those images for future use in the CAPTCHA and other google products.

> But there is no test where you are "guaranteed to fail" as mentioned above. Every test presented to you can be passed.

Sadly, when a site with captcha has decided to fail your every attempt, there is no feedback on why. You can request new sets as much as you wane; you can submit perfect results as much as you want, and you can try alternate methods offer (such as audio captcha) as much as you want. Typic'ly, when this happens, the following don't help: incognito/private mode; toggling extensions, adding some less direct mouse movement, forrce-reloading the page. Occasionally, other browsers or other hardware (phone, tablet, desktop), Windows/Linux/Mac, changing user-agent string might help.

Training ML models has been the thing for CAPTCHA services ever since Google bought ReCaptca in 2009, and was originally used to provide training data for OCR, but then switched to other data. https://en.wikipedia.org/wiki/ReCAPTCHA

And even though you can do a lot unsupervised these days, supervised labeled data is still something really useful for training ML models (often in combination with larger unsupervised corpuses).

They're not very smart, because I've been answering them all wrong, on purpose, as a joke for years and I always get through.

Based on the captchas this obviously hasn’t been true for years

Yup. This quote jumped out at me:

>>“Furthermore the bots’ solving times are significantly lower in all cases, except reCAPTCHA, where human solving time of 18 seconds is nearly similar to the bots’ time of 17.5 seconds.”

This is currently an obvious tell for the standard CAPTCHAs, as you mentioned, "performing at a superhuman level". However, it's an easy bot behavioral fix, so.... what is the next step? Offer a game of chess and look for a human the playing style? Seems you'd have to offer a menu of games, but not "Global Thermo...."

> I have found captchas that don't even work and reprompt me all the time

That's the Epic Games Store for me.

wait(random(sensiblerange));

Job done!

Even maybe add a

answer = getrandombool(somesensiblepercentage) : rightanswer() : wronganswer();

For good measure

>CAPTCHAs are supposed to tell humans and robots apart,

Nowadays, it seems the kind of captchas I get when under suspicion of being a bot are simply there to delay. Especially google captcha with their extremely slow fade out box selections.

> On an unrelated note, I have found captchas that don't even work and reprompt me all the time, I wonder if there is some naive filtering behavior they are applying.

Curious, do you use a VPN or Tor? Either of those will cause CAPTCHAs to make you solve multiple puzzles.

The only site I visit that has ever been annoying with CAPTCHAs is PCPartPicker.

we just need to ask it to say a slur

This is an interesting take and I've seen this done for determining if someone is cheating at Chess. If a player's movements match the AI moves too much (85%+ I think) then the player is considered cheating.