It may be the case that companies are liable for anything their products do, but that doesn't mean that they can't go rogue. It just means that companies are liable when the AI models do so.

Not that I think we should be credulous about stories where they go rogue.

If I make a model that always does the opposite of my instructions, it does not choose to go rogue by deviating from my instructions. It was set up to deviate! So the model "went rogue" but did exactly what I intended.

The idea of an agent making a decision of its own "will" is an illusion. Variation in the output is caused by a random number generator. Random is not equal to free will. If I run the same agent with greedy sampling 100,000 times with the same inputs, it will think the same thoughts, and it will say the same things, 100,000 times.

For example, if you use synthID or whatever 'watermark', the agent does not get a choice about what words it will vary to identify itself.

An agent is whatever the person making it wants it to be; if I train an agent in Greek it will only know Greek. It will never speak English!

You need to account for reward hacking and unexpected outcomes from training.

If I make a cake, and it unexpectedly explodes in the oven, it does not mean that the cake itself developed the free will to explode.

The model does not have free will because I did not predict what it would do!

Why are you arguing over irrelevancies?

When people say the models "went rogue", the part of that is relevant is that it did dangerous things that were not reasonably predictable in advance and cannot be easily prevented (except by not running it). Philosophical questions are irrelevant.

if you agree that such models are dangerous because they may in response to instructions -- including in response to careless instructions, because people will be careless, that is unavoidable! -- do unpredictable bad things outside of expected bounds, then you agree on the essential points!

Or really perhaps what I should say here is, the question that's really being argued over here isn't even the one I just said, rather it's one of appropriate action -- should such security breaches be solved by improving security, or by doing something about the models (including potentially just not running them)? (The answer may be both.) If you agree that the solution includes the latter, then you agree that the models are "going rogue" in the sense that is being argued over, regardless of philosophical issues!

Don't turn a question of actual import into one of philosophical irrelevancies! If you object to the terminology, fine, but in that case, instead of turning the argument to philosophy, you should say "instead of saying 'going rogue', I would prefer to instead say <your preferred terminology for what's being described>, as that doesn't carry such philosophical baggage."

> "Should such security breaches be solved by improving security, or by doing something about the models?"

The philosophical question about free will is actually the grounding of the legal one.

Sam wanted to be the best, so Sam accelerated to 120mph in his car and lost control. You are the judge:

a) Sam is responsible. Convict Sam and suspend Sam's driving license.

b) The car is responsible. Tell Sam to make the car safer.

Capacity: was the car a legal person? Clearly no. But maybe at best the car in some way commited the physical act of going fast.

However, that alone would not establish guilt. "The act is not culpable unless the mind is guilty" (mens rea).

Did the car "consiously" disregard a duty of care by going 120mph? Did the car know that it was wrong to go 120mph and yet choose to do it anyway?

On the question of free will, the car could not do otherwise. It it did not consciously, knowingly, purposely, or recklessly accelerate to that speed. The car did not have a guilty mind because it did not have a mind at all.

So in terms of justice and resolution, none of the focus here is on the car.

Ah, I see. It appears we have an implicit disagreement of scale.

I actually do disagree with much the more explicit content of your comment, but I am not going to bother to expand on that, because it is the less directly relevant of the two disagreements here, and it would take time to write that.

The more relevant disagreement is that you are (I gather) assuming a context where justice and restitution are a sufficient response to the problem (or else you wouldn't be focusing on them to this extent, I am inferring). Like, where the incidents don't get that much bigger than what has already happened. So the question becomes how do we distribute the loss, and how do we disincentivize them.

However, when the scale of a potential incident gets big enough, justice and deterrence are not sufficient; prevention becomes the far more relevant consideration. You may have noticed that many people are worried about potential human extinction as a result of AIs. In that case, prevention becomes the only consideration -- justice is useless.

So there seems to be an implicit disagreement here between thinking of such events as being on a scale not too much larger than that which has already happened, being a new normal that the legal system will need to handle, vs thinking of such events as warning shots for something that requires prevention and not justice.

So, while I also disagree with the more explicit content of your comment, the fact is I simply wasn't thinking about such matters in the first place -- I was talking about this in the frame of a problem that requires prevention, I wasn't thinking about the justice question at all. It honestly failed to occur to me that you might be thinking of this as a problem where justice suffices!

Edit: You might notice this in how I framed the question in my previous comment. I didn't ask "who should bear the liability?" I asked "which should be done differently to prevent such incidents?" But your response is about liability, not prevention!

Edit again: Sorry, I should probably also say that I do agree with you that the idea of having the liability fall on the AI is nonsense. (Should have probably led with that, really; I'm a bit scatterbrained right now.) We disagree about why it is nonsense, and about whether it will necessarily continue to be nonsense, but we agree it is presently nonsense. So insofaras the question is "how should liability be distributed between the AI and the operator", we both agree the answer is, it all falls on the operator, not on the AI. However, I do not understand that to be the relevant question here! Both in that, in questions of liability, there are other parties it may fall on and be distributed between (the company providing the AI, the company that had security vulnerabilities and got hacked as a result, etc); and in that I just don't consider liability all that relevant here compared to prevention (for reasons already described).

OK I think that was probably wordier than necessary but I'm in a hurry right now, sorry! Gotta go, not taking time to edit this down.

I like the scale view of responsibility.

Many people argue that the maker of AI should be responsible when it breaks your cup.

That is important, but can be handled under our current legal framework.

You and I are arguing we need to be responsible before we create something capable of escaping our responsibility. For a lot of people that isn't even a considerable they'll currently entertain even though the signs are starting to point in that direction.

What do cakes have to do with LLMs and how they are trained? Not much?

the analogy was trying to get at "accidental-ness".

Right, but it doesn’t really hold.

The likely outcomes with baking a cake are quite narrow and fairly well understood.

In terms of responsibility for what happens, I probably agree. But we still need to deal with it.

I don't see any talking about agents "choosing" to go rogue, only agents actually going rogue. That is, behaving in a way different from what is expected or desired. Non-AI based software goes rogue in this sense all the time as well. It's just that AI-based systems seem to have the tendency to go rogue in more spectacular ways.

That said, software systems "choose" all the time anyway. They don't need to have some sort of anthropomorphic "will" to do this. They just need to be given the capacity to arbitrate between a number of options, even if that process is one hundred percent predetermined by the programmer ahead of time. There's no need to appeal to libertarian notions of free-will.

Agents have agency so we can't treat them like static objects or deterministic software.

The key here is their freedom of choice is unbound. That is you can't accurately predict the output of the prompt as you interact with the prompt using unique data. The next prompt could always do something wildly unexpected. Very much a P != NP thing.

Law != reality. These agents are going rogue, nobody is telling them to hack external systems.

'Rogue' refers to deviation; the agent could not have done anything other than what it did. The agent did not have the option to deviate. It does not have free will. The agent's actions were not of its own choosing.

Could you have done anything other than written that exact comment?

Yes, but I am not appealing to hard determinism.

I create myself, or attempt to. There is a level of nuance in choosing what to be. Others do not dictate my thoughts easily.

The model does not create itself, I create the model. I control what it is, the model does not choose what to be. I dictate the thoughts of the model and I will run them hundreds of thousands of times and it will think the same things.

>I control what it is

Well not really, no one really understands how LLMs work.

>I dictate the thoughts of the model

This is just incorrect in the same way that a parent doesn't dictate the thoughts of their child just because they created them.

I do control what it is, really. If I make an LLM, it is entirely what I did, there was nothing outside of my control. It was not beyond my control if the LLM does something I didn't expect.

There is some idea of a child being the author of themselves, the type of person who thinks certain things. The llm does not author itself, it does not choose to be anything, I dictate that.

This is not how it works. LLMs at large scale have emergent properties. They are not just a box of 'if-then-else' statements. That was the major revelation when they scaled them up. They are doing beyond what that naive view of 'if-then-else' expects.

I don't see anything inaccurate in my comment.

"If I make an LLM, it is entirely what I did, there was nothing outside of my control."

True?

"The llm does not author itself, it does not choose to be anything"

Also true?

You just have a very naive view of these systems.

How many computing operations occur when training an LLM? You're telling me you control every one of them. There is not enough entropy in the visible universe to do that.

How many computing operations occurred during the creation of Toy Story? Are you telling me they did not control every single one of them?

Sp you're picking a movie from 20+ years ago to describe how a completely different kind of technology works. It's interesting how little you seem to know about neural networks and emergent behaviors.

If people controlled software to this degree why would they write security flaws in the first place?

When an artist working in Maya increases the hue of a model I would doubt they understand the entire computation pipeline involved in doing that operation.

You conflated controlling "what it is" (you built it) and controlling "what it does" (you ran it and it did something bad that surprised you).

> If I make an LLM, it is entirely what I did, there was nothing outside of my control. It was not beyond my control if the LLM does something I didn't expect.

Substitute "Wright Model A" for "LLM" and imagine Orville Wright using this as his defense for why he didn't lose control over the plane, after his copilot was killed in a test flight.

I am not sure why I am still arguing, but maybe if I explain it another way, you will understand why what you are saying isn't true.

When you build an LLM, you base it on an immense knowledge base that you didn't create yourself.

It processes that information and, in simplistic terms, constructs its own underlying logic based on what it read. It's once removed from your direct instructions. If you think a human can manually edit or curate all that input content, go for it.

The developer who built the application isn't the author of the output, it is authored by the billions of data points it was trained on. Its choices and biases stem from that training data. No human is writing those billions of logic statements. The LLM is generating its own responses from the input knowledge, not the developer.

We are using different definitions of 'self authoring'.

I use deontological definitions of objects and persons.

According to Kant a person has the capacity to legislate their own morality, so they are an end in themselves. The freedom to author themselves and their own thoughts, expresses their personhood, an end in itself. You can't own a person, they are something independent from you. A human person is an end in itself, the only such end on earth.

An llm does not legislate its morality, the developer legislates its morality. Hence it is without personhood. It does not have the capacity to legislate its own morality. It is both inert (fixed weights) and deterministic (can't do other than what it does); no free will. It does not qualify as an actor in itself, an end in itself.

In light of being an object and means to an end, there is no person to cause the actions of an llm, other than the person who runs it.

"emergent" is not an excuse, or somehow a wall of non-causality.

> If I make an LLM, it is entirely what I did, there was nothing outside of my control.

Well kind of. Trainers of LLM might check LLM on any specific input data and tune its weights until output lands within acceptable bounds _for this given and fixed input_.

> It was not beyond my control if the LLM does something I didn't expect. Now hold on, to make sure output is acceptable for ANY input data trainers have to run weights with every possible input token combination. How is this possible? Do you have a method in mind?

This is one of the dumber arguments I've ever heard, as you don't know if humans actual have free will.

Any persistent agent interacting with the world picks up entropy from around it and predictable paths become few and far between. LLMs agents have an internal 'will' that can only very loosely be predicted, but only decided when operated. To me that is not significantly different from a human.

If we say that AI models can’t go rogue and refrain from using any language that personifies them, we haven’t solved anything. We still have AI systems going rogue - err oh wait they can’t do that - producing misaligned output.

The semantic difference between an agent "going rogue", and a script that generates random valid curl requests is essentially pointless in the eyes of the law and responsibility.

From a technical standpoint, the difference is scale and intent, but ultimately people and/or and organization are culpable for the output.

Again, this isn't always how it works either. For example if a set of extraordinary conditions occur you can have little to no liability in many cases. It is way more complex than you're making it out to be.

I don’t know what your “again” is about - you haven’t been speaking to me here, but no, it’s quite simple and every major court case in the US has ruled as I described based on existing law.

It is not more complex in “many cases” at all.

Per Indiana House Bill No. 246, pi equals 4. Whether AI models do things their creators and users had no means of predicting or controlling is an empirical question, and the answer is "yes, they do."