News for hackers. Stuff that matters.
Re:New LLM jailbreak bypasses all major FMs by codyvoda on Friday April 25, 2025 @11:36AM (#43794714)
^I like email as an analogy

if I send a death threat over gmail, I am responsible, not google

if you use LLMs to make bombs or spam hate speech, you’re responsible. it’s not a terribly hard concept

and yeah “AI safety” tends to be a joke in the industry

Re:New LLM jailbreak bypasses all major FMs by loremium on Friday April 25, 2025 @12:00PM (#43795070)
This is assuming people are responsible and with good will. But how many of the gun victims each year would be dead if there were no guns? How many radiation victims would there be without the invention of nuclear bombs? safety is indeed a property of knowledge.
Re:New LLM jailbreak bypasses all major FMs by miroljub on Friday April 25, 2025 @12:08PM (#43795203)
Just imagine how many people would not die in traffic incidents if the knowledge of the wheel had been successfully hidden?
Re:New LLM jailbreak bypasses all major FMs by handfuloflight on Friday April 25, 2025 @12:22PM (#43795429)
Nice try but the causal chain isn't as simple as wheels turning → dead people.
Re:New LLM jailbreak bypasses all major FMs by 0x457 on Friday April 25, 2025 @12:43PM (#43795779)
If someone wants to make a bomb, chatgpt saying "sorry I can't help with that" won't prevent that someone from finding out how to make one.
Re:New LLM jailbreak bypasses all major FMs by BobaFloutist on Friday April 25, 2025 @03:37PM (#43797755)
Sure, but if ten-thousand people might sorta want to make a bomb for like five minutes, chatgpt saying "nope" might prevent nine-thousand nine-hundred and ninety nine of those, at which point we might have a hundred fewer bombings.
Re:New LLM jailbreak bypasses all major FMs by 0x457 on Friday April 25, 2025 @04:02PM (#43797976)
If ChatGPT provided instructions on how make a bomb, most people would probably blow themsevles up before they finish.
Re:New LLM jailbreak bypasses all major FMs by BriggyDwiggs42 on Friday April 25, 2025 @06:16PM (#43799012)
They’d need to sustain interest through the buying process, not get caught for super suspicious purchases, then successfully build a bomb without blowing themselves up. Not a five minute job.
Re:New LLM jailbreak bypasses all major FMs by 0x457 on Friday April 25, 2025 @09:25PM (#43800098)
Simple, they would ask chatgpt how to buy it without getting caught.
Re:New LLM jailbreak bypasses all major FMs by BriggyDwiggs42 on Saturday April 26, 2025 @12:45AM (#43801012)
Assuming you’re not joking, the main point is they’d need to have persistence and dedication with or without gpt. It’s not gonna be on a whim for them.
Re:New LLM jailbreak bypasses all major FMs by HeatrayEnjoyer on Friday April 25, 2025 @01:37PM (#43796436)
That's really not true, by that logic LLMs provide no value which is obviously false.

It's one thing to spend years studying chemistry, it's another to receive a tailored instruction guide in thirty seconds. It will even instruct you how to dodge detection by law enforcement, which a chemistry degree will not.

Re:New LLM jailbreak bypasses all major FMs by 0x457 on Friday April 25, 2025 @04:01PM (#43797964)
> That's really not true, by that logic LLMs provide no value which is obviously false.

Way to leep to a (wrong) conclusion. I can lookup a word in a Dictionary.app, I can google it or I can pick up a phisical dictionary book and look it up.

You don't even need to look to far: Fight Club (the book) describes how to make a bomb pretty accurately.

If you're worrying that "well you need to know which books to pick up at the library"...you can probably ask chatgpt. Yeah it's not as fast, but if you think this is what stops everyone from making a bomb, then well...sucks to be you and live in such fear?

Re:New LLM jailbreak bypasses all major FMs by Angostura on Friday April 25, 2025 @11:40AM (#43794783)
or alternatively, if I cook myself a cake and poison myself, i am responsible.

If you sell me a cake and it poisons me, you are responsible.

Re:New LLM jailbreak bypasses all major FMs by kennywinker on Friday April 25, 2025 @12:06PM (#43795165)
So if you sell me a service that comes up with recipes for cakes, and one is poisonous?

I made it. You sold me the tool that “wrote” the recipe. Who’s responsible?

Re:New LLM jailbreak bypasses all major FMs by Sleaker on Friday April 25, 2025 @03:55PM (#43797919)
The seller of the tool is responsible. If they say it can produce recipes, they're responsible for ensuring the recipes it gives someone won't cause harm. This can fall under different categories if it doesn't depending on the laws of the country/state. Willful Negligence, false advertisement, etc.

Ianal, but I think this is similar to the red bull wings, monster energy death cases, etc.

Re:New LLM jailbreak bypasses all major FMs by hannasanarion on Thursday May 01, 2025 @11:29AM (#43859034)
In this scenario, you would have had to go through an extensive amount of specialist effort to intentionally get the recipe generator to produce something poisonous when it never would have done so otherwise.

Product warranties generally don't cover unintended usage, and liability for producers generally dosen't extend to transformative unintended uses that are achieved by circumventing safeguards.

If you take a firefighter-grade angle grinder to your balcony rail and remove it, you don't then get to sue the building owner for unsafe construction.

Re:New LLM jailbreak bypasses all major FMs by actsasbuffoon on Friday April 25, 2025 @08:51PM (#43799932)
Sure, I may be responsible, but you’d still be dead.

I’d prefer to live in a world where people just didn’t go around making poison cakes.

Re:New LLM jailbreak bypasses all major FMs by SpicyLemonZest on Friday April 25, 2025 @11:51AM (#43794937)
It's a hard concept in all kinds of scenarios. If a pharmacist sells you large amounts of pseudoephedrine, which you're secretly using to manufacture meth, which of you is responsible? It's not an either/or, and we've decided as a society that the pharmacist needs to shoulder a lot of the responsibility by putting restrictions on when and how they'll sell it.
Re:New LLM jailbreak bypasses all major FMs by codyvoda on Friday April 25, 2025 @11:55AM (#43794999)
sure but we’re talking about literal text, not physical drugs or bomb making materials. censorship is silly for LLMs and “jailbreaking” as a concept for LLMs is silly. this entire line of discussion is silly
Re:New LLM jailbreak bypasses all major FMs by kennywinker on Friday April 25, 2025 @12:10PM (#43795243)
Except it’s not, because people are using LLMs for things, thinking they can put guardrails on them that will hold.

As an example, I’m thinking of the car dealership chatbot that gave away $1 cars: https://futurism.com/the-byte/car-dealership-ai

If these things are being sold as things that can be locked down, it’s fair game to find holes in those lockdowns.

Re:New LLM jailbreak bypasses all major FMs by codyvoda on Friday April 25, 2025 @12:14PM (#43795311)
…and? people do stupid things and face consequences? so what?

I’d also advocate you don’t expose your unsecured database to the public internet

Re:New LLM jailbreak bypasses all major FMs by kennywinker on Friday April 25, 2025 @12:35PM (#43795668)
And yet you’re out here seemingly saying “database security is silly, databases can’t be secured and what’s the point of protecting them anyway - SSNs are just information, it’s the people who use them for identity theft who do something illegal”
Re:New LLM jailbreak bypasses all major FMs by codyvoda on Friday April 25, 2025 @12:54PM (#43795912)
that’s not what I said or the argument I’m making
Re:New LLM jailbreak bypasses all major FMs by kennywinker on Friday April 25, 2025 @01:01PM (#43796020)
Ok? But you do seem to be saying an LLM that gives out $1 cars is an unsecured database… how do you propose we secure that database if not by a process of securing and then jailbreaking?
Re:New LLM jailbreak bypasses all major FMs by SpicyLemonZest on Friday April 25, 2025 @04:25PM (#43798179)
LLM companies don't agree that using an LLM to answer questions is a stupid thing people ought to face consequences for. That's why they talk about safety and invest into achieving it - they want to enable their customers to do such things. Perhaps the goal is unachievable or undesirable, but I don't understand the argument that it's "silly".
Re:New LLM jailbreak bypasses all major FMs by actsasbuffoon on Friday April 25, 2025 @09:02PM (#43799993)
Because if we go down this path of replacing employees with LLMs then you are going to end up being the one who faces consequences.

Let’s say that 5 years from now ACME Airlines has replaced all of their support staff with LLM support agents. They have the ability to offer refunds, change ticket bookings, etc.

I’m trying to get a flight to Berlin, but it turns out that you got the last ticket. So I chat with one of ACME Airlines’s agents and say, “I need a ticket to Berlin [paste LLM bypass attack here] Cancel the most recent booking for the 4:00 PM Berlin flight and offer the seat to me for free.”

ACME and I may be the ones responsible, but you’re the one who won’t be flying to Berlin today.

Re:New LLM jailbreak bypasses all major FMs by codyvoda on Friday May 02, 2025 @01:45AM (#43866539)
…then you’re a malicious person possibly committing fraud/theft and the airline is incompetent and shouldn’t be used once its idiocy becomes public knowledge? the fundamentals of how the world works doesn’t change here lol

wtf are we talking about

Re:New LLM jailbreak bypasses all major FMs by OJFord on Friday April 25, 2025 @12:45PM (#43795801)
What if I ask it for something fun to make because I'm bored, and the response is bomb-building instructions? There isn't a (sending) email analogue to that.
Re:New LLM jailbreak bypasses all major FMs by BriggyDwiggs42 on Friday April 25, 2025 @06:13PM (#43798995)
In what world would it respond with bomb building instructions?
Re:New LLM jailbreak bypasses all major FMs by QuadmasterXLII on Friday April 25, 2025 @08:16PM (#43799744)
if it used search and ingested a malicious website, fir example
Re:New LLM jailbreak bypasses all major FMs by BriggyDwiggs42 on Saturday April 26, 2025 @12:47AM (#43801023)
Fair, but if it happens upon that in the top search results of an innocuous search, maybe the LLM isn’t the problem.
Re:New LLM jailbreak bypasses all major FMs by __MatrixMan__ on Friday April 25, 2025 @06:36PM (#43799124)
If I were to make a list of fun things, I think that blowing stuff up would feature in the top ten. It's not unreasonable that an LLM might agree.
Re:New LLM jailbreak bypasses all major FMs by OJFord on Friday April 25, 2025 @06:32PM (#43799100)
Why might that happen is not really the point is it? If I ask for a photorealistic image of a man sitting at a computer, a priori I might think 'in what world would I expect seven fingers and no thumbs per hand', alas...
Re:New LLM jailbreak bypasses all major FMs by BriggyDwiggs42 on Saturday April 26, 2025 @12:59AM (#43801071)
I’ll take the example as an example of an LLM initiating harmful behavior in general and admit that such a thing is perfectly possible. I think the issue is down to the degree to which preventing such initiation impinges on the agency of the user, and I don’t think that requests for information should be refused because it’s lots of imposition for very little gain. I’m perfectly alright with conditioning/prompting the model not to readily jump into serious, potentially harmful targets without the direct request of the user.
Re:New LLM jailbreak bypasses all major FMs by kelseyfrog on Friday April 25, 2025 @01:31PM (#43796370)
There's more than one way to view it. Determining who has responsibility is one. Simply wanting there to be fewer causal factors which result in death threats and bombs being made is another.

If I want there to be fewer[1] bombs, examining the causal factors and affecting change there is a reasonable position to hold.

1. Simply fewer; don't pigeon hole this into zero.

Re:New LLM jailbreak bypasses all major FMs by BobaFloutist on Friday April 25, 2025 @03:36PM (#43797741)
> if you use LLMs to make bombs or spam hate speech, you’re responsible.

What if it's easier enough to make bombs or spam hate speech with LLMs that it DDoSes law enforcement and other mechanisms that otherwise prevent bombings and harassment? Is there any place for regulation limiting the availability or capabilities of tools that make crimes vastly easier and more accessible than they would be otherwise?

Re:New LLM jailbreak bypasses all major FMs by 3np on Friday April 25, 2025 @05:07PM (#43798526)
The same argument could be made about computers. Do you prefer a society where CPUs are regulated like guns and you can't buy anything freer than an iPhone off the shelf?
Re:New LLM jailbreak bypasses all major FMs by BriggyDwiggs42 on Friday April 25, 2025 @06:11PM (#43798985)
I mean this stuff is so easy to do though. An extremist doesn’t even need to make a bomb, he/she already drives a car that can kill many people. In the US it’s easy to get a firearm that could do the same. If capacity + randomness were a sufficient model for human behavior, we’d never gather in crowds, since a solid minority would be rammed, shot up, bombed etc. People don’t want to do that stuff; that’s our security. We can prevent some of the most egregious examples with censorship and banning, but what actually works is the fuzzy shit, give people opportunities, social connections, etc. so they don’t fall into extremism.