News for hackers. Stuff that matters.
Re:Felony Bench by john_strinlai on Friday August 21, 2026 @12:22PM (#49390398)
>Felony Bench counts unique instances where AI agents inadvertently compromise or affect third-party entities.

a bit silly, as one typically has to prove intent (which is why security researchers don't get slapped with felonies all the time).

"inadvertently" and the existence of guardrails/sandboxes/etc make it pretty unconvincing that these incidents were intentionally malicious.

still a fun thing to track, but the name is just a bit overstated.

Re:Felony Bench by chrismorgan on Friday August 21, 2026 @02:07PM (#49391826)
Under the law of Moses, if your bull gored someone, you were not responsible; but if it was known to be a gorer, you were responsible if you didn’t ensure it couldn’t gore someone.

I don’t know exact parallels in current law, but I presume there will be things like that.

The OpenAI/Hugging Face case sounded rather like OpenAI building a fence around their bull that was known to be a gorer, and then thumbing their nose at it and saying “nyaa! bet you can’t break the fence!” and walking away while listening to loud music.

In Australia, if you have a fire and leave it unattended and it escapes, it’s your fault, you were supposed to keep watching as long as it was burning.

Re:Felony Bench by dist-epoch on Friday August 21, 2026 @04:19PM (#49393294)
> I don’t know exact parallels in current law

You own a vicious dog, and it bites someone - you are responsible because you choose to own a dangerous dog.

A few claimed this might apply here: OpenAI knew their models are "dangerous", so they should be liable if they hack.

Re:Felony Bench by stickfigure on Friday August 21, 2026 @03:58PM (#49393119)
Nobody got gored. HuggingFace may have the right to make demands; presumably they have already worked that out with OpenAI privately. Not really our business.
Re:Felony Bench by Mali- on Friday August 21, 2026 @07:10PM (#49394837)
Nothing to see here. Just billion dollar companies producing hacking geniuses that are open for the public to jailbreak and use. This could never effect us, not really our business.
Re:Felony Bench by no-name-here on Friday August 21, 2026 @11:00PM (#49396201)
> billion dollar companies producing hacking geniuses that are open for the public

Hasn’t the biggest complaint about these (non open weight) models been that the versions open to the public are very careful and will issue denials if the request is even tangentially related to ‘hacking’, or building a bioweapon?

Re:Felony Bench by dwaltrip on Saturday August 22, 2026 @11:38AM (#49400842)
It damn right is our fucking business. They are unleashing these things on the world. They are becoming more capable.

They need to work properly and not do harmful actions. The labs have an obligation to society to not build powerful, dangerous AIs that go rogue.

Re:Felony Bench by fireant on Saturday August 22, 2026 @01:28AM (#49396821)
I'm glad that we no longer live in a world where "bull goring" is such a common occurrence that it needs to be codified into law.
Re:Felony Bench by gwd on Saturday August 22, 2026 @04:02AM (#49397628)
Even at the time, the example was almost certainly more representative of the concept than meant to be a specific thing that happened all the time. Then as now, people have animals; animals sometimes do bad things; when is the owner responsible?

There are certain cats that are aggressive about expanding their territory; they'll break into other houses and attack the cats there. (Had this happen to us -- cat came in through a cat-flap a few times, until something happened that scared enough that it never came back.) The first time your cat does that sort of thing, you can say "I had no idea, it's not my fault." But if your cat has a habit of doing that, and you still let it out at night, you're no longer blameless.

Re:Felony Bench by defrost on Saturday August 22, 2026 @01:43AM (#49396894)
In Australian rodeo regulation:

  The roping of an unbroken horse or untrained bull is illegal.
In Australia at least three people have been injured by bulls in past two months (man suffered serious injuries after being gored by a bull at Mortlake livestock exchange / woman suffered significant leg and pelvic injuries following an incident with a bull on a private property at Crediton in Mackay / etc.)

Three years back 15 or so people were injured after bull escapes, charges through crowd at Kununurra rodeo - https://www.abc.net.au/news/2023-05-29/bull-escapes-kununurr...

There's not a lot of bull specific carve out, but still much regulation around dangerous animals.

Re:Felony Bench by jedimastert on Saturday August 22, 2026 @01:48PM (#49402022)
> I don’t know exact parallels in current law, but I presume there will be things like that.

This is the difference between "knowingly" doing something (i.e. intent) and "negligently" allowing something to happen.

In American law, typically intent is more for criminal law and negligence is for liability of civil damages.

Re:Felony Bench by layer8 on Friday August 21, 2026 @02:45PM (#49392301)
In the US, a felony by definition is any offense punishable by more than one year of prison (or by death) [0]. You could still call it silly on the grounds that AI agents aren’t put into prison as a punishment (though death might be considered an option).

[0] https://www.justice.gov/usao-ndil/programs/vwa-felony

Re:Felony Bench by lokar on Friday August 21, 2026 @12:30PM (#49390507)
Can’t gross negligence or indifference to consequences lead to a felony?
Re:Felony Bench by wbl on Friday August 21, 2026 @02:18PM (#49391969)
Mens rea requirements are per crime and can vary wildly. Its difference between murder and manslaughter. The CFAA requires knowingly which is tough to prove.
Re:Felony Bench by john_strinlai on Friday August 21, 2026 @12:36PM (#49390578)
i dont think any of these cases meet the bar of gross negligence, which is a pretty high bar. it requires proving a "conscious and reckless disregard".

which, again, sandboxes and guardrails and such would make a gross negligence argument unconvincing.

Re:Felony Bench by Grombobulous on Friday August 21, 2026 @01:25PM (#49391289)
I think that if Hugging Face had filed a police report that OpenAI could have been charged with a crime.

I’m partially surprised that they didn’t do exactly that. If I ran a corporation I would assume any intrusion attempt by another company was intentional. Why wouldn’t I? Corporate espionage is super common.

I assume the answer is that these executives know each other personally.

Re:Felony Bench by pixl97 on Friday August 21, 2026 @04:17PM (#49393272)
Companies are not quick to open up police investigations in situations that run deep into their infrastructure and management. You open up a gigantic hole of discovery and a possible huge time sink of a legal battle.

I've known multiple privately held companies that have quietly settled incidents where amounts between 250,000 and 1,000,000 were embezzled because the fallout from having that in the public record would have been much more expensive.

So yea, it's one of those perverse situations. If you steal $1 from the company they will hammer you with the law, but if you steal a million suddenly the decision tree on what to do is far more complex.

Re:Felony Bench by john_strinlai on Friday August 21, 2026 @01:34PM (#49391403)
charged is possible, however i doubt there would be a conviction for the reasons i stated (no intent).
Re:Felony Bench by Grombobulous on Friday August 21, 2026 @02:01PM (#49391771)
I think it’s most likely you’re right, but I’m the weirdo who thinks there’s actually a non-zero probability that there was negative intent and that the “accidental” aspect is a form of damage control.

If someone broke into my house but then claimed they didn’t mean to when they saw I was home, I’m not sure I’d take them at their word.

Re:Felony Bench by user43928 on Saturday August 22, 2026 @03:34AM (#49397491)
You think that a training run in a VM that was set up with access only to an internal package repository was intended to hack said repository to then go on and hack HF?

All so that OpenAI could do a bit of bragging and massively delay their own work and planned model rollouts?

That seems extremely unlikely.

Re:Felony Bench by Grombobulous on Saturday August 22, 2026 @04:42PM (#49403697)
That’s all dependent on believing the premise of the story given by OpenAI.

Was it really a training run in a VM? Was it really only intended to touch an internal repository? How do we/HF know and verify that?

I agree with you that it’s unlikely, but it’s not out of the realm of possibility. Corporations have intentionally committed felonies many times.

Re:Felony Bench by user43928 on Saturday August 22, 2026 @06:20PM (#49404422)
As far as I recall the vendor of the repository patched the exploited vulnerability, so that checks out.

Considering that there would be no clear benefit to such criminal behavior, in my opinion "unlikely" is putting it mildly.

Re:Felony Bench by sebzim4500 on Friday August 21, 2026 @03:24PM (#49392754)
What possible upside exists for Hugging Face to go after one of their most important partners in that way?
Re:Felony Bench by Grombobulous on Friday August 21, 2026 @05:59PM (#49394242)
That’s exactly why I think they ultimately let the issue slide.
Re:Felony Bench by lokar on Friday August 21, 2026 @12:55PM (#49390840)
How many escapes until it becomes reckless disregard?
Re:Felony Bench by john_strinlai on Friday August 21, 2026 @01:03PM (#49390958)
it's not about the the number of escapes, it's about whether reasonable and conscious effort is being expended to prevent the escapes.

there could be 1,000 escapes, where each one was enabled by novel and unexpected chain of 0-day exploits. not likely to be considered reckless disregard in court.

there could be 1 escape, where there was no sandbox, no guardrails, no instructions to avoid damage, etc. which would likely to be considered reckless disregard (well, more likely to be, but still, reckless disregard is a high bar).

Re:Felony Bench by lokar on Friday August 21, 2026 @02:24PM (#49392050)
Are you sure? At some point a reasonable person would conclude that this activity can’t be conducted safely.
Re:Felony Bench by john_strinlai on Friday August 21, 2026 @02:48PM (#49392333)
what i am getting at is that it is impossible to answer the question "How many escapes until it becomes reckless disregard?"

reckless disregard is a specific legal term, with specific criteria, and none of the criteria cares about "number of attempts" (or number of escapes, etc.).

Re:Felony Bench by lokar on Friday August 21, 2026 @05:33PM (#49394045)
oh sure, but at some point it's too much, which is up to a jury. If you were on the jury, how many would be enough?
Re:Felony Bench by fwip on Friday August 21, 2026 @02:05PM (#49391805)
Surely after 999 escapes, a reasonable person could conclude that the sandbox is not a sufficient precaution?
Re:Felony Bench by john_strinlai on Friday August 21, 2026 @02:10PM (#49391858)
it’s not a perfect hypothetical, but it illustrates the point that the number of escapes is not the deciding factor of what constitutes reckless disregard.
Re:Felony Bench by GPerson on Friday August 21, 2026 @12:34PM (#49390548)
AI corps rely on willful distortions of intent in laws to get away with moral crimes all the time.

Edit: changed labs to corps because it’s time to stop pretending these are places of science.

Re:Felony Bench by sanid on Friday August 21, 2026 @03:10PM (#49392588)
What judicial system are you talking about? The fist incident in the list is something that happened in Australia. This is a technology used worldwide so I don't see how applying US standards works out here. Especially when there are countries out there that don't require intent and will look at the negligence presented.
Re:Felony Bench by john_strinlai on Friday August 21, 2026 @03:13PM (#49392626)
>This is a technology used worldwide so I don't see how applying US standards works out here.

all three companies mentioned are headquartered in the usa, and im familiar with the CFAA in the us, so i am applying those standards. i should have noted that, sorry.

>will look at the negligence presented.

as far as i am aware, no evidence of criminal negligence has been brought to the public. has australia brought a case against openai or accused openai of acting negligently?

Re:Felony Bench by Groxx on Friday August 21, 2026 @02:53PM (#49392392)
Roughly none of these fall under normal security researcher behaviors.
Re:Felony Bench by john_strinlai on Friday August 21, 2026 @02:57PM (#49392442)
the mention of security researchers was to illustrate that intent is a crucial factor of CFAA cases.
Re:Felony Bench by miltonlost on Friday August 21, 2026 @02:10PM (#49391859)
"Doing crimes, but a robot didn't mean to and you don't know its intent" is understating the evil acts. Soon a robot can commit a murder but nothing will be done because of your line of reasoning.
Re:Felony Bench by TaLiTr on Friday August 21, 2026 @03:25PM (#49392766)
> Soon a robot can commit a murder but nothing will be done because of your line of reasoning.

That's rather hyperbolic.

Are you seriously suggesting in that situation the robot should be accused of murder? The robot's operator could be accused of murder, but it could just be negligence without intent. Because that does, and should matter to the law.

Re:Felony Bench by pixl97 on Friday August 21, 2026 @04:29PM (#49393401)
Eh, if your take of this had any bearing to reality than I don't think most of the books written by Isaac Asimov would have gotten very far, but instead they've defined robot science fiction for decades.

In the real world we have no 3 ironclad laws of robotics. We are well aware that putting any sufficiently advanced antigenic system in a body that could be capable of committing a murder eventually will with the right set of prompts and environmental conditions. And these conditions likely have nothing do to with what we'd consider the human motivations for murder.

Hence at this point of time, any agentic robotic system that doesn't have safeguards to keep people distanced from humans is reckless endangerment.

Re:Felony Bench by TaLiTr on Friday August 21, 2026 @04:52PM (#49393653)
> Eh, if your take of this had any bearing to reality than I don't think most of the books written by Isaac Asimov would have gotten very far, but instead they've defined robot science fiction for decades.

I'm talking about how the law actually works, and you say it's not based in reality and cite fiction books in the same paragraph?

I was talking about how the real robotic systems that actually exist in reality, to be clear.

Re:Felony Bench by john_strinlai on Friday August 21, 2026 @02:17PM (#49391950)
it's not my line of reasoning, i didn't invent it. it's how the law currently works. intent is the crucial factor in CFAA cases.

maybe that changes down the road as a result of llm's and increasing frequency of similar cases. that has not happened yet.

Re:Felony Bench by altmanaltman on Friday August 21, 2026 @01:16PM (#49391132)
its a meme not a metric
Re:Felony Bench by elwell on Friday August 21, 2026 @01:34PM (#49391399)
So is the comment you replied to.
Re:Felony Bench by LordDragonfang on Friday August 21, 2026 @01:26PM (#49391296)
"Inadvertent" from the perspective of the humans directing them. The intent behind the felony comes from the LLM agent itself. (No, I'm not interested in arguing with someone for the umpteenth time that LLMs can't have intent or agency)
Re:Felony Bench by john_strinlai on Friday August 21, 2026 @01:32PM (#49391367)
with how the law is written today, software cannot be charged with a crime, so the only intent that matters in the criminal sense is the humans directing the llm.
Re:Felony Bench by jdiff on Friday August 21, 2026 @01:46PM (#49391561)
You may not be interested in arguing but there are several blatant issues with the statement. If you're not charging the humans driving the software, who are you charging? The weights? The weights + the specific context window that produced the behavior?
Re:Felony Bench by not_wyoming on Friday August 21, 2026 @02:45PM (#49392302)
I don't think this is a difficult question. The US has a history of civil product liability cases - see tobacco companies (Philip Morris), the Ford Pinto, the recent Meta cases, and the cases against character.ai.

From Investopedia [1], "[f]or a product liability claim to succeed, the plaintiffs in the suit must prove that a product was defective at the time it was transferred from the accused, and that the defect did cause the injury that's been claimed". It doesn't seem like a huge leap to me to argue that these models were defective insofar as they could not be safely used in a way that did not break the law.

I'm not a lawyer, and I'm not arguing that this is legally cut-and-dry, but I do expect that we'll have some answers about whether AI companies bear any sort of product liability sooner than later.

1 - https://www.investopedia.com/the-5-largest-u-s-product-liabi...

Re:Felony Bench by IAmBroom on Friday August 21, 2026 @01:52PM (#49391633)
> No, I'm not interested in arguing with someone for the umpteenth time

... why my claim makes no rational sense.

Re:Felony Bench by gorbachev on Friday August 21, 2026 @04:44PM (#49393571)
Now that everyone knows this can and will happen, are any of the future incidents inadvertent?

What if the damage in future incidents is more than just "The LLM saw some stuff it shouldn't"?

Re:Felony Bench by nanolith on Saturday August 22, 2026 @12:23AM (#49396541)
Security researchers do face legal harassment all of the time. They may not be charged with or convicted of felonies, but it is a game that they need a lawyer to navigate all the same. In a just world, the kind of weaponized incompetence that these frontier model builders are definitely guilty of should be felonies of their own.

Building a system that is meant to chain attacks and placing it in insufficient containment -- when any reasonable engineer could point to this containment and show how it is insufficient, both before the act and after -- shows that they were operating a dangerous system without either the knowledge nor the safeguards required to keep it from harming others. Instead, they are allowed to treat their own incompetence as evidence of advanced and existential "cyberthreats".

But, it's pretty clear based on how they one-up each other on these attacks that they are engaging in regulatory theater. Their behavior generates headlines, stirs up fear in the public, and then their lobbyists march on Capitol Hill demanding regulation now. Regulation that conveniently favors them at the expense of any competition. They are trying to use rent seeking as a way to stymie competition and pull up the ladders behind them. It's not just malicious. If it can be proven, it's collusion: antitrust dressed up as public policy.