It's actually pretty interesting to see how the captchas have evolved as, presumably, Google decides "OK, we have enough data to consistently identify this thing" and moves on to the next challenge.

I recall the modern (non-text) captchas used to be cars pretty much every time. Then, the images started getting grainier as they apparently wanted to improve their recognition in different conditions. Then crosswalks and store fronts became quite common, eventually with the same kinds of noise distorting images. Now I've started seeing things like buses, bridges, motorcycles, bicycles, etc. It feels like they've finished getting enough data for improving Google Maps and have begun moving towards collecting data for their self-driving car projects.

Google: spins off robotic car company Waymo, announcing public-facing robotaxi service launch in 2018

Also Google: "Our standard for 'what a machine couldn't possibly do' is identifying a stop sign."

Machines have been able to beat captchas for years and years. The point is that it costs money and that's good enough to prevent free and scalable abuse.

Isn't the captcha data used specifically to train that AI? That's what they did with the old one at least where it was used to train machines to read.

Those really grainy images, they always seemed like they had to have noise added on purpose? It was like really badly processed film grain, but if the images were enlargements then wouldn't they be pixelated?

Stuff you get now often requires cultural information, like "sidewalk" isn't a cross-cultural name, I'd guess almost everyone knows it, but meh. What classes as a store, is a lawyers office a store? Also, I seem to recall I had "click on all minivans"?? Not sure what one of those is, nor really what is classed as a car in USA, is an MPV a car [I'd guess that's what a minivan is?]? Do pedestrian crossing lights (green/red man) count as [part of] traffic lights? I've often wanted a short description of the locus of the terms they're using. Of course it never tells you if you failed, just gives you a further captcha, which it might have done anyway.

The fire hydrant one always annoyed me - in many countries stand-up fire hydrants are nonexistent or at least look very different. I guess they're banking on people having seen enough American movies/TV shows?

It's not like the word 'fire hydrant' exists in those countries either, so some kind of knowledge/learning/look-up is involved anyway, no?

In the UK we call them fire hydrants and they’re typically placed underground with a small metal cover in the middle of the road and a sign on the pavement (sidewalk for Americans) saying where it is.

Remember this belongs to Google https://patents.google.com/patent/US9407661B2/en

Are you sure? On the page it says:

    Current Assignee: Juniper Networks Inc

Was it owned by Google at some point in the past?

Yeah, a lot of the storefront ones are just plan hit things randomly until it lets you through - how am I supposed to know if a building with some writing on it in Korean is a store or something else? Are you supposed to include the poles in traffic lights or not?

The V2 was just annoyingly badly designed because the questions were badly put.

North American food is what amused me, I was thinking how many are going to get that one right outside North America.

> Then, the images started getting grainier as they apparently wanted to improve their recognition in different conditions.

I'd always assumed that was noise carefully tuned to throw off one machine learning model or another that was being used to beat the captcha, sort of like this: https://www.theverge.com/2017/11/2/16597276/google-ai-image-...

I think it might be the same when they switch to other types of objects (like crosswalks or bikes). Someone's model got too good, so they had to change to something else. I also get the impression that they add delays to the tile refresh before they do that.

I suspect Google now uses robots to generate captchas for humans, under the assumption their image recognizers are far better than anyone else's. They already have some very well ones for other products (self driving cars, street view) and lots of street-level city imagery. That would explain why their captchas are so difficult for humans to solve -- they're testing if you see things like their "AI," not like other humans.

>I'd always assumed that was noise carefully tuned to throw off one machine learning model or another that was being used to beat the captcha

I was thinking it was trying to dirty up the image just like the lenses on cameras get dirty. What happens to the image recognition when there's water spots, dirt, mud, etc on the lens that keeps parts of the image obscured?

The images get a lot grainier when you browse over VPN or fail the first round. So I suspect it's meant to actually throw off captcha-solvers.

what would be the training data value in getting humans to classify images you have clean versions of, after applying simulated noise?

If you have the clean version of the image, you need to get that classified by a human - then you can throw noisy versions of it into the training set for your AI. You don’t need to ask a human, hey, I added noise to a picture of a yield sign. Is it still a picture of a yield sign?

Yep.

With the possibility of almost uniquely identifying us on the web through fingerprinting... Google, of all companies is in the perfect position to know that my web request was made by me... And therefore I'm not a robot.

You can only conclude that recaptcha is a ml training exercise.

>You can only conclude that recaptcha is a ml training exercise.

They're not secretive about it

https://developers.google.com/recaptcha/

> With the possibility of almost uniquely identifying us on the web through fingerprinting... Google, of all companies is in the perfect position to know that my web request was made by me... And therefore I'm not a robot.

The article explains that this is part of what reCAPTCHA does, e.g.:

> Finally they combine all of this data with their knowledge of the person using the computer. Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things. When you click that checkbox, Google reviews your browser history to see if it looks convincingly human.

But your point is otherwise right in that it's used for ML training, which Google admits as another commenter pointed out.

> Finally they combine all of this data with their knowledge of the person using the computer. Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things.

Human [n]: Entity that uses Google®-brand services.

— Google Dictionary, 2020 edition

Maybe, maybe not. From time to time google forces me to prove I'm not a robot because of unusual search queries. Either my queries are unusual or I have an infected computer. The thing is, it also happens to me on my iPhone.

Or, maybe they feel I'm not pulling my own weight, seeing as I rarely ever click on adds. They probably need more monkeys to feed the beast so I get selected to train their AI beast.

Edit: It could also be that I'm always running on incognito mode.

I wouldn't be surprised if soon enough you'll have to play a mini-match of Starcraft 2 as part of REcaptcha to train DeepMind.

And I find it equally interesting to think about what happened to Google Books project after half the world was entering text into captchas perhaps close to a decade? Is the project still going on?

Eventually you will have to park a self driving car in a tight spot for captcha.

Helping scan all the world's books as part of a plan to make them freely available to all is much easier to get behind than tuning map objects for their self driving cars. Particularly when most books that were scanned never, ever showed up on Google books.

> Particularly when most books that were scanned never, ever showed up on Google books.

If one looks at the history of Google Books one can see that they started with big ambitions, but hit copyright in quite intensive ways. That also changed their approach to other projects. Clearing all rights internationally isn't easy.

OTOH, succeeding at self-driving cars has the potential to save lives by the thousands.

And if they succeed to much they can monopolize transportation and thus mobility. Not a power I want to have in a private company. Especially not in a company from outside my country's jurisdiction. Where I can't have an impact via democratic law making process.

> Especially not in a company from outside my country's jurisdiction. Where I can't have an impact via democratic law making process.

This is false. EU governments have already placed significant restrictions and fines upon US tech companies in the past. There is no reason to believe that they won't be able to again.

Those fines are the perfect representation of the lack of control EU has over US tech giants. They are essentially opaque and impossible to inflence through normal regulatory and political channels so the only options are the big guns. You can be sure Google is not going to heavily invest in the EU tech sector under such adversarial set-up. There's a subtle blackmail here: we push the envelope as far as it goes and the EU can choose to submit or risk technological backwardness.

It's a great situation for the US economy but a very bad strategical position for Europe.

Don't forget the bonus: all your forum accounts can be linked to your google account, just in case you didn't use your gmail as the email address.

That's what really kills me here. This is lock-in at least as bad as the Equifax situation.

Self-destructing cookies is a solution

Not if Google has collected enough other identifying information about you to determine whether or not you're a robot.

But I'm kinda hoping that the reason I keep having to identify cars and store fronts is that my refusal of third-party cookies is causing them to have no idea who I am. But that might be the optimistic view.

In any case, I wouldn't mind if sites stopped using recaptcha.

Identifying text for public domain works that Google is making available online is much different than forcing the web to train their models for their profit.

they stopped using the text because some forum campaign that promoted typing cursewords instead of the unkown word. they probably started showing cursewords on the rendered search highligths on google books.

there was a decent write up from a whitehat showing the damage, but I can't find it

Cite?

I can't imagine some forum has enough traffic to meaningfully screw up their data, and they don't tell you which of the two words is the unknown word, so you're just going to fail a lot doing that.

It was very obvious which was which and you had a 50/50 chance. I can confirm this used to work and I always used a curse word plus the known word.

It was pretty big on 4chan at least at some point in the early 2010's. And the unknown word was always the hard one to read iirc.

This was extremely common on 4chan maybe 7 or so years back I can't remember the exact year and it worked for a very long time before anything was done about it and everyone knew to do it. Google asked for two words and presented them as two different fonts. The real word that needed transcribing was always identifiable and you could just write what you wanted as long as you got the second test word correct. Much fun was had.

Sounds like something 4chan would advocate.

More importantly, it's trivial to filter out everyone putting in the same swear.

4chan was doing it specifically with a six letter racial slur that's unrepeatable.

Since you're so afraid to use the word 'nigger' here's a book by a Harvard law professor you should probably read: https://en.wikipedia.org/wiki/Nigger:_The_Strange_Career_of_...

I guessed correctly: the author belongs to the one racial group for whom it's politically correct to use the word.

Better safe than sorry, there are enough people who think context doesn't matter.

So what's going to happen if you write "nigger" that you're so unwilling to write it? It's not like you'd be calling someone a nigger, you'd be using it in a clearly informative manner,

My understanding is that some folks have gotten in trouble anyway. Good luck to you!

I don't care to find out.

not really. it started with the word "penis" for the original "campaign" but since a single word was obviously ineffective the meme improved.

> they stopped using the text because some forum campaign that promoted typing cursewords instead of the unkown word. they probably started showing cursewords on the rendered search highligths on google books.

If I remember correctly, Google later on also sometimes showed two "known" words or, if they had actual other evidence that you are human, two unknown words.

Isn't there a good OSS alternative for ReCaptcha?

Captchas inherently require some security by obscurity; they're not a good fit for OSS solutions.

I'm doubtful about any good solution for having computers decide whether we are sufficiently human.

> I'm doubtful about any good solution for having computers decide whether we are sufficiently human.

Two questions: - Couldn't a computer just temporarily hire a human to prove there is a human involved? - Why are we using recaptcha or verifying humanity anyway? I understand stopping spam, scams, and fraud, but scraping already public data doesn't present significant harm.

Yeah, it shouldn't be used to limit public data. The main use is to prevent spammers from spamming fora or registering thousands of throwaway email addresses.

The Turing test. Have one of your ops chat with them to see if they’re a bot.

Cheaper: have a robot chat with them ...

Google Duplex?