That might have made sense in a pre-LLM world. People need to recognize the liability of letting an LLM access the internet and act on their behalf, because that liability exists for someone.

Still, that characterization falls under negligent or reckless depending on if the person knew or should have known the actual danger. It is different than intent.

Then who gets prosecuted?

In legal tradition, if there's not a law you broke, you can't be prosecuted for it.

(Yes, I'm aware of numerous historical exceptions. Those exceptions are traditionally considered not ideal.)

> In legal tradition, if there's not a law you broke, you can't be prosecuted for it.

That’s incorrect.

If there is not a law the prosecutor or plaintiff can point to and say you broke, you can’t be prosecuted.

We wouldn’t need much legal process after a prosecution was initiated if it was impossible to prosecute without a law actually being broken.

Who do you expect to get prosecuted when no law has been broken?

You might need to rewatch A Man For All Seasons.

Maybe the developer of the software that didn't add basic authorization checks on the cancel reservation route should be fined, forced to provide a refund to their customer, etc

Referring to the first link from the page: https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gy...

So far, no one. It's like prosecuting an accident.

So what? Everyone in this chain is knowingly and intentionally developing or using an unreliable tool…

I'm not sure: if you know that LLMs are prone to crime, using them and not checking in enough to trigger 'knowingly' might be gross negligence?

Define crime though, because one particular action could be both a crime and not, depending on a range of factors that the LLM might not be aware of.

Even having a million legal experts on call weighing in on every prompt/response will not agree on everything.

Even things like "go and break into this system, use whatever means you need to" might not be a crime.

Keeping a vicious dog doesn't have to lead to a crime either, but that doesn't mean you are not responsible, if something happens.

Legal responsibility can be in forms (e.g., civil tort liability) other than criminal.

In the scale of mental states in crime, negligence of any kind is several steps below knowing/intentional; you can't be liable for an intentional crime because of mere negligence of any degree.

You could be liable for the (civil) tort of negligence, though.

Details depend. And setting up negligence and being willfully ignorant is often not something the courts see as a defense.

OpenAI and Anthropic both have currently safety teams that look for misbehavior in their models (and to some extent, voluntarily disclose what they find to the public). Going forward, it would be hard for them to argue they don’t know their models do stuff like this.

Yes, which is the grey area. "Can / might do" vs "they trained it to do that explicitly" is, I believe, the grey area - whether or not they're the same thing.

Intent matters for a lot of this - and "intent" is a pretty strong, well discussed legal term.

Your honor, my LLM spun the turbines real fast, as a practical joke!

It's not your intent to use a tool that can cause real world damage, to actually do such damage.

It was your negligence in that case. A different legal concept than intent.

To clarify I was referring to Stuxnet here (and the current wave of critical infrastructure hacks, which now have "haha whoops the matmul went a bit funny!" as plausible deniability).

> knowingly

Intent or negligence.

But you could also use this to argue in the other way to say that they are using due care and therefore not negligent

Well, if you’re writing reports saying “we know our model only decides to commit felonies 0.001% of time which we judge to good enough to deploy” … I’m not sure that gets you off the hook the hook for the felonies.

It works for gun or even car manufacturers. They know that some sales will be used for crime.

So if I port scan the internet without knowing it's going to be illegal, I'm legally covered?

As they say, ignorance of the law is [generally] no excuse; "knowingly" and "intentionally" here are about knowing what you're doing and meaning to do it, rather than whether you know it's illegal.

This section of the USC is about false ID offenses, but it discusses culpable states of mind generally. I think the context helps illustrate it though.

https://www.justice.gov/archives/jm/criminal-resource-manual...

----

> A knowing state of mind with respect to an element of the offense is (1) an awareness of the nature of one's conduct, and (2) an awareness of or a firm belief in the existence of a relevant circumstance, such as the "stolen," the "produced without lawful authority," or "false" nature of the identification document. The knowing state of mind requirement may be satisfied by proof that the actor was aware of a high probability of the existence of the circumstance (e.g., stolen or false nature of the document), although a defense should succeed if it is proven that the actor actually believed that the circumstance did not exist after taking reasonable steps to ensure that such belief was warranted.

> As we pointed out in United States v. United States Gypsum Co., 438 U.S. 422, 445 (1978), a person who causes a particular result is said to act purposefully if `he consciously desires that result, whatever the likelihood of that result happening from his conduct,' while he is said to act knowingly if he is aware `that the result is practically certain to follow from his conduct, whatever his desire may be as to that result.

----

This Congressional Research Service Report discusses mens rea further, including a brief mention of the CFAA. The whole thing is worth a read if you're interested in the topic.

https://www.congress.gov/crs-product/R46836

----

> The approach largely reflected in the MPC and some federal precedent is to distinguish between "intention" or purpose on the one hand as being limited to a conscious object or desire, and "knowledge" on the other hand as capturing a requirement of awareness of a high probability or to a practical certainty.

> The Supreme Court in Bailey referenced this distinction approvingly and suggested that intention or purpose "corresponds loosely with the common-law concept of specific intent, while 'knowledge' corresponds loosely with the concept of general intent." Some federal courts utilize a definition of "knowing" that approximates the MPC approach, instructing that to act knowingly a defendant must have "realized what he was doing and [be] aware of the nature of his conduct" rather than acting "through ignorance, mistake or accident."

> Congress has also signaled an intent to distinguish between the two mens rea terms in this way in particular statutes. For instance, prior to 1986, the Computer Fraud and Abuse Act (CFAA) proscribed "knowingly" accessing a computer without authorization or exceeding authorized access in certain circumstances. In its 1986 amendments, however, Congress changed the standard from "knowingly" to "intentionally," and the Senate report emphasized that the change was meant to require "more than that one voluntarily engaged in conduct . . . . Such conduct . . . must have been the person's conscious objective."

----

(Note, for reference, what requires a "knowing" vs. "intentional" state of mind in the CFAA: <https://www.law.cornell.edu/uscode/text/18/1030>)

The Justice Manual also has some relevant detail (the rest of this page is also worth a look, as it addresses the practical (and nominal) matter of what is and isn't likely to be prosecuted (IANAL though, and I should stress that I'm not speaking to whatever might be the true realities of how the CFAA is applied):

https://www.justice.gov/jm/jm-9-48000-computer-fraud

----

> In either a "without authorization" case or an "exceeds authorized access" case, the attorney for the government must be prepared to prove that the defendant knowingly accessed a computer or area of a computer to which he was not allowed access in order to obtain or alter information stored there, and not merely that the defendant subsequently misused information or services that he was authorized to obtain from the computer at the time he obtained it.

> As part of proving that the defendant acted knowingly or intentionally, the attorney for the government must be prepared to prove that the defendant was aware of the facts that made the defendant’s access unauthorized at the time of the defendant’s conduct. Such an awareness could potentially be proven through various means, including the presence of technology intended (however unsuccessfully) to limit unauthorized access; written or oral communications sent to the defendant that unambiguously informed him that he is not authorized to access a protected computer or particular areas of it; or the defendant’s own statements or behaviors reflecting knowledge that his actions were unauthorized.

> Experience has demonstrated that in the large majority of "exceeds authorized access" cases brought by the Department, the operator of the computer system made some technological effort to protect the information at issue, thereby signaling the importance or sensitivity of that information. It is not necessary that this technological effort erect an impenetrable "technological barrier" or that the technology succeed in its intended purpose of preventing access. To the contrary, when the CFAA is violated, the technology all too often "permits" the defendant’s illegal access, often despite network defenders’ unsuccessful technological attempts to prevent it.

----