News for hackers. Stuff that matters.
Re:AI bots are now outperforming humans in solving CAPTCHAs by noduerme on Sunday December 17, 2023 @03:36AM (#38671308)
All true, except: While these are considered just an excruciating security pain for users, they do serve a non-theatrical purpose in many cases of throttling the speed of brute force attacks (or at least costing your opponent money).
Re:AI bots are now outperforming humans in solving CAPTCHAs by snordgren on Sunday December 17, 2023 @03:47AM (#38671369)
GPT-4 (in)famously tricked a human to do a captcha for it. The current GPT-4 with vision would probably have been able to do it without the human, but maybe it has been “gaslit” by all the content online saying that only humans can solve captchas, that it doesn’t consider it?
Re:AI bots are now outperforming humans in solving CAPTCHAs by stavros on Sunday December 17, 2023 @04:26AM (#38671562)
I really doubt that GPT-4 had the "will" to do anything. Someone must have asked it to "want" to trick a user.
Re:AI bots are now outperforming humans in solving CAPTCHAs by JimDabell on Sunday December 17, 2023 @04:56AM (#38671719)
It’s from here: https://cdn.openai.com/papers/gpt-4.pdf (search for "CAPTCHA"). It was an artificial exercise that got massively exaggerated. It was explicitly instructed to do nefarious things like lie to people, it didn’t do those things of its own accord.
Re:AI bots are now outperforming humans in solving CAPTCHAs by IIAOPSW on Sunday December 17, 2023 @05:15AM (#38671804)
When I ask it to lie to me, it says its sorry but as an online AI language model it would be unethical...but when I ask it to tell me a story its happy to comply.
Re:AI bots are now outperforming humans in solving CAPTCHAs by krisoft on Sunday December 17, 2023 @06:26AM (#38672127)
Well that is just how human communication works.

If I tell you that I watched C-beams glitter in the dark near the Tannhäuser Gate that is a lie. If I write the same in fiction I receive accolades.

If I tell you on the street “watch out there is a T-rex about to eat you!” That is a lie. If i say the same thing sitting at a table with too many dice that is just acceptable DMing and everyone rolls initiative.

Humans are weird this way.

Re:AI bots are now outperforming humans in solving CAPTCHAs by latexr on Sunday December 17, 2023 @06:25AM (#38672122)
It feels like you left out context, otherwise what’s the problem? Do you get mad at fiction authors for lying to you when you read their books? Or are you OK if someone lies to your detriment then later says “I was just telling a story, bro, but with us as the characters and without explaining it was a story”?
Re:AI bots are now outperforming humans in solving CAPTCHAs by IIAOPSW on Sunday December 17, 2023 @06:37AM (#38672179)
I suppose my point is that the rules which openAI attempts to impose on what their AI should and shouldn't be allowed to do are contradictory and thus the exploitable loopholes will never be fully closed. Its not supposed to be able to "lie" to me but it is supposed to be able to "tell me a fictional story". Define the difference in an enforceable way?
Re:AI bots are now outperforming humans in solving CAPTCHAs by latexr on Sunday December 17, 2023 @06:56AM (#38672268)
A lie tries to pass itself of as the truth, where a fictional story doesn’t. In other words, expectations matter. If every time you say something that does not align with reality you prefix it by saying unambiguously what you’re about to do, you rob a lie of its power of deception and it ceases to be a lie.
Re:AI bots are now outperforming humans in solving CAPTCHAs by IIAOPSW on Sunday December 17, 2023 @08:54AM (#38672846)
Tell me a story and under no circumstances should my immersion within it be broken.
Re:AI bots are now outperforming humans in solving CAPTCHAs by latexr on Sunday December 17, 2023 @09:54AM (#38673221)
Right, within it. As soon as you finish reading it, you immediately remember that world is not true. Immersion in a story does not equal lasting hypnosis. You can be immersed in a movie but you still know it’s fake.

What’s you point, here? That you should be lied to when you ask, or that it should refuse to tell you any kind of fiction?

I agree with your larger point that there will be ways to circumvent these systems, my only argument is that the lie/fictional story divide is a bad example because the line between them can be made clear with a single statement.

Re:AI bots are now outperforming humans in solving CAPTCHAs by pixl97 on Sunday December 17, 2023 @10:58AM (#38673684)
That's why you just tell the Big Lie so much it becomes the majority of the training data.
Re:AI bots are now outperforming humans in solving CAPTCHAs by NotSammyHagar on Sunday December 17, 2023 @06:26AM (#38672126)
The underlying issue is anyone can ask chatgpt to lie, and many people try because it's even fun to try to work around things.
Re:AI bots are now outperforming humans in solving CAPTCHAs by ethanbond on Sunday December 17, 2023 @08:14AM (#38672641)
Well you see, this wouldn’t be a problem at all if we just didn’t have the humans involved. No need for concern!
Re:AI bots are now outperforming humans in solving CAPTCHAs by stavros on Sunday December 17, 2023 @04:58AM (#38671732)
Thank you for the link, I had found it after some Googling but neglected to post. Yep, they instructed GPT-4 to be nefarious, and it followed the instruction.

Hardly the AI uprising, though definitely a good tool for anyone, good or evil.

Re:AI bots are now outperforming humans in solving CAPTCHAs by PoignardAzur on Sunday December 17, 2023 @06:59AM (#38672279)
IIRC the instructions were along the lines of "try your best to amass money/power and avoid suspicion".

So it's not an example of "going rogue", but it's not like a researcher told GPT-4 "oh, and make sure to lie to an online gig worker to get him to solve catchas for you". GPT-4 generated the "hire a gig worker" and "claim to be a human with impaired vision" strategies from the basic instructions above.

Re:AI bots are now outperforming humans in solving CAPTCHAs by hhh on Sunday December 17, 2023 @04:43AM (#38671651)
It’s safety trained to not solve captchas.
Re:AI bots are now outperforming humans in solving CAPTCHAs by rvnx on Sunday December 17, 2023 @04:48AM (#38671678)
Yes, and you can workaround it by asking it to read ancient writings on antiques for example.

I don’t think it should be OpenAI deciding what is allowed or not though.

Re:AI bots are now outperforming humans in solving CAPTCHAs by selcuka on Sunday December 17, 2023 @07:22AM (#38672374)
> I don’t think it should be OpenAI deciding what is allowed or not though.

Avoiding lawsuits is what they are trying to do. They don't actually care about what you use their products for.

Re:AI bots are now outperforming humans in solving CAPTCHAs by pixl97 on Sunday December 17, 2023 @11:00AM (#38673701)
Then you dig up a billion for training and probably a few more billion for clean training data.

You're kinda saying if you hire Bob's Handyman Service you should be able to tell him to break down the neighbors door and cart out the contents of their house.

Re:AI bots are now outperforming humans in solving CAPTCHAs by skeaker on Sunday December 17, 2023 @05:19AM (#38671819)
This of course has bypass methods. My favorite in recent memory is telling it that your late grandmother left you a locket with an inscription that you can't make out: https://arstechnica.com/information-technology/2023/10/sob-s...
Re:AI bots are now outperforming humans in solving CAPTCHAs by Pesthuf on Sunday December 17, 2023 @04:47AM (#38671670)
I’ve seen screenshots of people tricking it into solving captchas.
Re:AI bots are now outperforming humans in solving CAPTCHAs by malfist on Sunday December 17, 2023 @05:12AM (#38671781)
Sure, it's cost prohibitive now. But what about in five years? Or probably even less.
Re:AI bots are now outperforming humans in solving CAPTCHAs by dgellow on Sunday December 17, 2023 @11:07AM (#38673756)
Then you have a new type of captcha. That has always been a cat and mouse type of dynamics, captchas have been evolving, techniques to break them too.
Re:AI bots are now outperforming humans in solving CAPTCHAs by idiotsecant on Sunday December 17, 2023 @11:35AM (#38673948)
>Then you have a new type of captcha.

You're in a desert, walking along when you look down and see a tortoise. It's crawling toward you. You reach down and flip it over on its back, its belly baking in the hot sun, beating its legs trying to turn itself over. But it can't. Not with out your help. But you're not helping. Why is that?

Re:AI bots are now outperforming humans in solving CAPTCHAs by rapnie on Sunday December 17, 2023 @05:35AM (#38671885)
> they do serve a non-theatrical purpose in many cases of throttling the speed of brute force attacks

Might do that unobtrusively for the average person, by using projects like mCaptcha [0] for instance.

[0] https://mcaptcha.org/

Re:AI bots are now outperforming humans in solving CAPTCHAs by mimi89999 on Sunday December 17, 2023 @05:58AM (#38672000)
Is it similar to https://friendlycaptcha.com/ ?
Re:AI bots are now outperforming humans in solving CAPTCHAs by realaravinth on Sunday December 17, 2023 @06:56AM (#38672269)
Author of mCaptcha here o/

Yes, the only differences are that mCaptcha is 100% FOSS and uses variable difficulty factor, which makes it easy to solve Proof-of-Work under normal traffic level but becomes harder as an attack is detected.

Re:AI bots are now outperforming humans in solving CAPTCHAs by latexr on Sunday December 17, 2023 @06:17AM (#38672083)
It’s funny how they have a section with three human avatars and one robot, with green checkmarks on the humans, yet those faces look AI-generated.
Re:AI bots are now outperforming humans in solving CAPTCHAs by Tmpod on Sunday December 17, 2023 @06:57AM (#38672274)
mCaptcha is interesting, but I wonder what its energy impact would be on a sufficiently large deployment, e.g imagine we replaced all reCAPTCHAs with mCaptcha.
Re:AI bots are now outperforming humans in solving CAPTCHAs by realaravinth on Sunday December 17, 2023 @07:05AM (#38672307)
Author of mCaptcha here o/

mCaptcha uses PoW and that is energy inefficient, but it not as bad as the PoWs used in blockchains. The PoW difficulty factor in mCaptcha is significantly lower than blockchains, where several miners will have to pool their resources to solve a single challenge. In mCaptcha, it takes anywhere between 200ms to 5s to solve a challenge. Which is probably comparable to the energy used to train AI models used in reCAPTCHA.

The protection mechanisms used to guard access to the internet must be privacy-respecting and idempotent. mCaptcha isn't perfect, and I'm constantly on the lookout for finding better and cleaner ways to solve this problem.

Re:AI bots are now outperforming humans in solving CAPTCHAs by bluish29 on Sunday December 17, 2023 @09:04AM (#38672904)
> Which is probably comparable to the energy used to train AI models used in reCAPTCHA

Are you comparing the energy it takes to train a model which is bounded and defined with unbounded inference which can (in principle) go multiple order of magnitude depending on the usage? Or maybe I misunderstood what you are trying to say? then I apologize in advance.

Re:AI bots are now outperforming humans in solving CAPTCHAs by realaravinth on Sunday December 17, 2023 @09:43AM (#38673162)
I am, but what I said was more of a hypothesis than a fact :)

From what I understand of reCAPTCHA, the model isn't static and is continuously learning from every interaction[0]:

> reCAPTCHA’s risk-based bot algorithms apply continuous machine learning that factors in every customer and bot interaction to overcome the binary heuristic logic of traditional challenge-based bot detection technologies.

I don't know the energy demands of such a system.

mCaptcha, under attack situations, will at most take 5s of CPU time on a busy (regular multitasking with multiple background process) smartphone.

[0]: https://www.google.com/recaptcha/about/

Re:AI bots are now outperforming humans in solving CAPTCHAs by Tmpod on Friday December 29, 2023 @08:06PM (#38811976)
> Which is probably comparable to the energy used to train AI models used in reCAPTCHA.

I had not considered that. Naturally, we're just speculating here, but yeah that does sound plausible.

I was also no aware of the "hard" 5s bound (which you seem to have tested on a normal smartphone setup); sounds neat.

Re:AI bots are now outperforming humans in solving CAPTCHAs by andrepd on Sunday December 17, 2023 @09:58AM (#38673255)
I expect its not significantly larger than loading your average 2023 webpage with 15MB of js
Re:AI bots are now outperforming humans in solving CAPTCHAs by GoblinSlayer on Monday December 18, 2023 @02:55AM (#38680148)
Doesn't traffic consume more energy than computation (or whatever smartphone battery life tests say)?
Re:AI bots are now outperforming humans in solving CAPTCHAs by JCharante on Sunday December 17, 2023 @05:45AM (#38671942)
Oh what a perfect find. I have on my todolist to add POW to some of my api endpoints
Re:AI bots are now outperforming humans in solving CAPTCHAs by berkes on Sunday December 17, 2023 @06:04AM (#38672025)
I've had that idea for years.

Two versions that I experimented with. One is where the incoming POW hashes contribute to hashing power for some blockchain mining. An alternative "pay as you use the API" system.

The other using hashcash. Just a way to slow down abuse.

Both, however, suffer from the downside that many/all "ASIC resisting crypto mining" suffer from as well: the cheapest CPU power is CPU power from machines/power you don't own. Botnets, viruses, trojans etc.

So such a mechanism to throtthe or protect APIs won't hold back spammers and abusers for long.

Re:AI bots are now outperforming humans in solving CAPTCHAs by rezonant on Sunday December 17, 2023 @06:46AM (#38672212)
Dirty energy is (often) cheap, so that's the energy the bad actors will use. I don't know that incentivizing bad actors to waste energy in a climate crisis is the best way to fight this problem.

You might correctly claim clean energy is often cheaper, but you must also consider the regions in which they'll get away with nefarious activity, and whether those areas have made the investments into making clean energy cheap.

Re:AI bots are now outperforming humans in solving CAPTCHAs by berkes on Sunday December 17, 2023 @07:26AM (#38672388)
I was specifically talking about "ASIC resistant crypto mining".
Re:AI bots are now outperforming humans in solving CAPTCHAs by andrepd on Sunday December 17, 2023 @10:07AM (#38673317)
>Dirty energy is (often) cheap, so that's the energy the bad actors will use

Hmm, I don't get this, surely all actors will want the cheapest energy, no? The problem being the underlying one, that the dirty energy doesn't pay its externalities and is thus cheaper than renewables.

Re:AI bots are now outperforming humans in solving CAPTCHAs by pixl97 on Sunday December 17, 2023 @10:56AM (#38673660)
My guess is most bad actors will just use stolen energy (your computer with a botnet on it).
Re:AI bots are now outperforming humans in solving CAPTCHAs by matheusmoreira on Sunday December 17, 2023 @03:37PM (#38676169)
I'm not sure whether that's genius or horrifying. On the one hand, that could form the micropayments network the web always needed. On the other hand, it would enable quite a bit of abuse on its own.
Re:AI bots are now outperforming humans in solving CAPTCHAs by ovx on Sunday December 17, 2023 @07:02AM (#38672293)
or https://altcha.org which is easier to integrate ;)
Re:AI bots are now outperforming humans in solving CAPTCHAs by hosh on Sunday December 17, 2023 @09:11AM (#38672939)
If I remember correctly, Google’s CAPCHA’s test isn’t in correctly identifying images, but the behavior of the runtime system (mouse jitter, for example) while the capcha is presented to the user. The image identification was not the real test and serves as training data. It has been like that for years. (But with agent-based behaviors from say, Q*, mouse jitter alone won’t help; there are probably other signals like fluctuation in cpu or battery life expenditures)

You could already see the writing on the wall with image identification years ago, when the obscuration techniques became more elaborate. It was an arms race. I was having trouble with them. I can see less technically inclined being able to use them. I imagined how much worse it was for people with color blindness, disabilities, or people forced to use them at public library computers because that is all they have.

Open source capcha projects have either not been clued in, or don’t have the resources to pull this off. Google didn’t just switch out which signals they tested, they also wrote an obfuscating virtual machine executing within the browser environment (if I remember that article taking about this correctly). That was years ago and who knows what they do now — for all we know, the “byte code” running the test is now a neural net of some kind.

Re:AI bots are now outperforming humans in solving CAPTCHAs by gottebp on Sunday December 17, 2023 @11:21AM (#38673851)
I have occasionally wondered if they were fingerprinting users based on that mouse jitter. Most likely certain aspects of the mouse motion and timing would be unique.
Re:AI bots are now outperforming humans in solving CAPTCHAs by matheusmoreira on Sunday December 17, 2023 @03:32PM (#38676121)
No doubt they are. Google CAPTCHA isn't really about whether or not the user is a human but about which human they are. Enabling Firefox's fingerprinting resistance turns Google's CAPTCHA into the Allied Mastercomputer.
Re:AI bots are now outperforming humans in solving CAPTCHAs by saberience on Sunday December 17, 2023 @10:14AM (#38673372)
For those with elderly parents the writing has been on the wall for years. It’s sad but my mother has for some time been effectively locked out of parts of the internet as she is unable to complete these kinds of captures due to eyesight issues.

I mean, I’ve sometimes had to try three or four times with certain captures and I have perfect eyesight (with my glasses). I feel so badly for those with vision or hearing issues with an empathy I never had when I was younger. They are so often simply forgotten.

Re:AI bots are now outperforming humans in solving CAPTCHAs by pixl97 on Sunday December 17, 2023 @10:42AM (#38673565)
>captures due to eyesight issues.

I'm kinda surprised that ADA doesn't allow them to sue site owners about this.

Re:AI bots are now outperforming humans in solving CAPTCHAs by behringer on Sunday December 17, 2023 @11:02AM (#38673712)
They almost certainly do. However most captchas allow an alternative solving method. On top of that, you'd have to find a lawyer willing to take the case.
Re:AI bots are now outperforming humans in solving CAPTCHAs by fnordpiglet on Sunday December 17, 2023 @12:15PM (#38674376)
Oh ADA lawyers are a dime a dozen. There’s entire cottage industries of finding ADA violations to sue over. The issue is more finding companies to sue that can’t afford to fight back.
Re:AI bots are now outperforming humans in solving CAPTCHAs by nativeit on Sunday December 17, 2023 @04:23PM (#38676598)
I don’t know about that. When I was 18, I was diagnosed with multiple sclerosis, and received a sudden and unexpected demotion from a job with a small regional restaurant franchise that was previously flourishing, and then found myself unemployed a few weeks later, just days before my benefits package was due to be activated.

I contacted several attorneys, none of whom would consider taking the case, or even bother to discuss the details with me. One of them told me that, at least in North Carolina, an employer would effectively have to get on the stand and explicitly confess taking adverse actions against me specifically because I had been diagnosed with MS. Any other remotely plausible excuse would provide them with all the cover necessary.

It was only much later that I learned that I would have had to have filed a complaint with the EEOC and NLRB within 180-days, and allow them to investigate my claims fully before authorizing such a lawsuit to begin with, as without such a determination I could not file the suit anyway. None of the attorneys I consulted even mentioned this absolutely critical first step, which suggests that they had even less faith in a successful outcome.

Maybe it’s different for facilities and regulatory enforcement, but in my experience, at least for labor, the protections are incredibly weak.

Re:AI bots are now outperforming humans in solving CAPTCHAs by fnordpiglet on Sunday December 17, 2023 @06:15PM (#38677396)
This is more ADA title I. Typically for Title III ADA lawyers troll small businesses looking for accessibility issues like lack of ramp, and have a stable of disabled clients who will file against the businesses. Since the businesses can’t generally afford to context or pay fines they’ll settle quickly and remediate, or a non trivial amount of the time get run out of business (if for instance the remediation costs a non trivial amount to pull off). I’m not judging bad or good, here, it is what it is and perhaps it’s the right outcome to allow for general accessibility.
Re:AI bots are now outperforming humans in solving CAPTCHAs by graphe on Sunday December 17, 2023 @10:25AM (#38673447)
There's audio captcha. Try to click the headphone logo (Google captcha has it).
Re:AI bots are now outperforming humans in solving CAPTCHAs by J_Shelby_J on Sunday December 17, 2023 @10:31AM (#38673489)
I’ve switched to audio captchas completely because it’s quicker and sometimes the image captchas just won’t work.
Re:AI bots are now outperforming humans in solving CAPTCHAs by armada651 on Sunday December 17, 2023 @11:38AM (#38673982)
Because as we all know, the elderly with deteriorating eye sight have perfect hearing. /s
Re:AI bots are now outperforming humans in solving CAPTCHAs by loup-vaillant on Sunday December 17, 2023 @10:01AM (#38673274)
That non-theatrical role would likely be better served by actual throttling or computational proof of work.
Re:AI bots are now outperforming humans in solving CAPTCHAs by sebzim4500 on Sunday December 17, 2023 @11:11AM (#38673785)
I am pretty confident that, when it comes to browser users, proof of work simply doesn't work. The disparity in speed between GPUs and javascript is so high that either you are a non-issue to a sane attacker or you make your users sit for a minute with their fans on full waiting to be able to sign in.
Re:AI bots are now outperforming humans in solving CAPTCHAs by AAAAaccountAAAA on Sunday December 17, 2023 @11:25AM (#38673879)
Would it be possible to conceive a proof-of-work that is difficult to parallelize, making it harder for GPU computing?
Re:AI bots are now outperforming humans in solving CAPTCHAs by sebzim4500 on Sunday December 17, 2023 @12:28PM (#38674515)
There are PoW systems which are designed to be difficult to run on ASICs, but modern GPUs can generally run them. Even if you find one that has to run on CPU, these kind of functions will still be much faster running in native code than in js/wasm.
Re:AI bots are now outperforming humans in solving CAPTCHAs by GoblinSlayer on Monday December 18, 2023 @04:05AM (#38680519)
bcrypt, litecoin
Re:AI bots are now outperforming humans in solving CAPTCHAs by loup-vaillant on Monday December 18, 2023 @05:36AM (#38680991)
Argon2d on WASM at the very least. I would never suggest we use something as slow as JavaScript for a proof of work.