ie "remove the squid from the novel All Quiet on the Western Front"
Particularly since, in this case, it's the alignment focused company (Anthropic) that's claiming it's creating AI agents that will go after humans.
For example, we have certain films to thank for an escalation in the tactics used by bank robbers which influenced the creation of SWAT which in turn influenced films like Heat and so on.
Is there a reason to think this was caused by the popularity of the films and not that it’s a natural evolution of the cat-and-mouse game being played between law enforcement and bank robbers? I’m not really sure what you are specifically referring to, so apologies if the answer to that question is otherwise obvious.
(The movie inspired reality, not the other way around.)
https://melmagazine.com/en-us/story/north-hollywood-shootout
But your point still stands, because it goes both ways.
> Gang leader Robert Sheldon Brown, known as “Casper” or “Cas,” from the Rollin’ 60s Neighborhood Crips, heard about the extraordinary pilfered sum, and decided it was time to get into the bank robbery game himself. And so, he turned his teenage gangbangers and corner boys into bank robbers — and he made sure they always brought their assault rifles with them.
> The FBI would soon credit Brown, along with his partner-in-crime, Donzell Lamar Thompson (aka “C-Dog”), for the massive rise in takeover robberies. (The duo ordered a total of 175 in the Southern California area.) Although Brown got locked up in 1993, according to Houlahan, his dream took hold — the takeover robbery became the crime of the era. News imagery of them even inspired filmmaker Michael Mann to make his iconic heist film, Heat, which, in turn, would inspire two L.A. bodybuilders to put down their dumbbells and take up outlaw life.
In the real world voice interfaces work terribly unless you have something sentient on the other end.
But people saw the movies and really really really wanted something like that, and they tried to make it.
It's one of many unfortunate anthropomorphic buzz words which conveniently wins hearts and minds (of investors) over to this notion that we're tickling the gods, rather than the more mundane fact that we're training tools for synthesising and summarising very, very large data sets.
It's quite reasonable to think that LLMs might plan and have thoughts of their own. No one understands consciousness or the emergent behavior of these models to say with much certainty.
It is the "Chinese room" fallacy to assume it's not possible. There's a lot of philosophical debate going back 40 years about this. If you want to show that humans can think while LLMs do not, then the argument you make to show LLMs do not think must not equally apply to neuron activations in human brains. To me, it seems difficult to accomplish that.
Of course, some would argue the Chinese Room is conscious.
0: https://www.sciencealert.com/quantum-entanglement-in-neurons... 1: https://www.scientificamerican.com/article/how-migrating-bir...
One can say the notion of free will is an unpacked bundle of near infinite forces emerging in and passing through us.
precisely, mathematically identical to infinite precision .. "yes".
Meanwhile, in the real world we live in it's essentially physically impossible to stage two seperate systems to be identical to such a degree AND it's an important result that some systems, some very simple systems, will have quite different outcomes without that precise degree of impossibly infinitely detailed identical conditions.
See: Lorenz's Butterfly and Smale's Horshoe Map.
Clearly you're aware of this, however I find that the majority of casual referrers to determinism are not.
More likely means that our brains are wired to see faces.
Are humans not also mixing a bag of experiences and coming up with a response? What's different?
A lot. Like an incredible amount. A description of a thing is not the thing.
There is sensory input, qualia, pleasure & pain.
There is taste and judgement, disliking a character, being moved to tears by music.
There are personal relationships, being a part of a community, bonding through shared experience.
There is curiosity and openeness.
There is being thrown into the world, your attitude towards life.
Looking at your thoughts and realizing you were wrong.
Smelling a smell that resurfaces a memory you forgot you had.
I would say the language completion part is only a small part of being human.
Just because it’s not persistent doesn’t mean it’s not there.
Like, I’m sort of inclined to agree with you, but it doesn’t seem like it’s something uniquely human. It’s just a matter of degree.
Living creatures have fundamental impetus to grow and reproduce that LLMS and AIS simply do not have currently. Not only that but animals have a highly integrated neurology that has billions of years of being tune to that impetus. For example the ways that sex interacts with mammalian neurology is pervasive. Same with need for food, etc. That creates very different neural patterns than training LLMS does.
Eventually we may be able to re-create that balance of impetus, or will, or whatever we call it, to make sapience. I suspect we're fairly far from that, if only because the way LLMs we create LLMs are so fundamentally different.
You can't prove humans do either. You can see how many times actual people with understanding something that's written for them. In many ways, you can actually prove that LLMs are superior to humans right now when it comes to understanding text.
Emphasis mine.
No, I don't think you can, without making "understanding" a term so broad as to be useless.
You can do this trick as well. Haven't you ever been to a class that you didn't really understand, but you can give correct answers?
I've had this somewhat unsettling experience several times. Someone asks you a question, words come out of your mouth, the other person accepts your answer.
But you don't know why.
Here's a question you probably know the answer to, but don't know why:
- I'm having steak. What type of red wine should I have?
I don't know shit about Malbec, I don't know where it's from, I don't know why it's good for steak, I don't know who makes it, how it's made.
But if I'm sitting at a restaurant and someone asks me about wine, I know the answer.
You said nothing meaningful that couldn't also have been spat out by an LLM. So? What IS then the secret sauce? Yes, you're a never resting stream of words, that took decades not years to train, and has a bunch of sensors and other, more useless, crap attached. It's technically better but, how does that matter? It's all the same.
For a LLM, language is their whole world, they have no body to care for, just stories about people with bodies to care for. For them, as opposed to us, language is first class and the rest is second class.
There is also a difference in scale. LLMs have been fed the entirety of human knowledge, essentially. Their "database" is so big for the limited task of text generation that there is not much left for creativity. We, on the other hand are much more limited in knowledge, so more "unknowns" so more creativity needed.
intention is what exactly? It's the set of options you imagine you have based on your belief system, and ultimately you make a choice from there. That can also be replicated in LLMs with a well described system prompt. Sure, I will admit that humans are more complex than the context of a system prompt, but the idea is not too far.
It is a duck of quacks like a duck.
Humans with their tiny amount of data and "special sauce" can produce much more consistent results even though they may be giving the objectively wrong answer. They can also tell you when they don't know about a certain topic, rather than lying compulsively (unless that person has a disorder to lie compulsively...).
More and more researches are showing via brain scans that we don’t have free will. Our subconscious makes the decision before our “conscious” brain makes the choice. We think we have free will but the decision to do something was made before you “make” the choice.
We are just products of what we have experienced. What we have been trained on.
Maybe yours isn't, but mine certainly is. Intelligence is an emergent property of systems that get good at prediction.
LLMs have a million plans and a million thoughts: they need to simulate all the characters in their text to complete these texts, and those characters (often enough) behave as if they have plans and thoughts.
Compare https://gwern.net/fiction/clippy
Just because you can give a reductionist explanation to a phenomenon, it doesn't mean that it's the best explanation.
Your objection boils down to "sure you're right, but there's more to it, man"
So, what more is there to it?
Unless there is a physical agent that receives its instructions from an LLM, the prediction that the OP described is correct.
It's not very helpful to point that out, especially if you can't do it with specifics so that people can correct themselves and move closer to the truth.
Your contribution is destructive, not constructive.
Maybe there should be a stronger filter on the input considering these things don’t have any media literacy to understand cautionary tales. It seems like a bad idea to continue to feed it stories of bad behavior we don’t want replicated. Although I guess anyone who thinks that way wouldn’t be in the position to make that decision so it’s probably a moot point.
Not necessarily correct if you consider agent architectures where one LLM would come up with a plan and another LLM executes the provided plan. This is already existing.
That goal was incepted by a human but I don’t see that as really mattering. We’re this AI given access to a machine which could synthesize things and a few other tools it might be able to act in a dangerous manner despite its limited form of will.
A computer doing something heinous because it is misguided isn’t much better than one doing so out of some intrinsic malice.
No. That hugely multidimensional vector maps to much higher abstractions than words.
We are talking about deep learning models with hundreds of layers and trillions of parameters.
They learn patterns of reasoning from data and learn a conceptual model. This is already quite obvious and not really disputed. What is disputed is how accurate that model is. The emulation is pretty good but it's only an emulation.
But I must push back, there really seem to have been no incidences where something like emergent behaviour has been observed. They're able to generate text fluently, but are dumb and unaware at the same time, from day one. If someone really thinks they've solid evidence of anything other than this, please show us.
This is coming from someone who has watched commentary on quite a sizeable number of stockfish TCEC chess games over the last five years, marvelling in the wonders of thie chess-super-intelligence. I am not against appreciating amazing intelligences, in fact I'm all for it. But here, while the tool is narrowly useful, I think there's zero intelligence, and nothing of that kind has "emerged".
Of course it's concerning, or at the very least, it's relevant! We get tied up in these debates about motives, experiences, what makes something human or not, etc., when that is less relevant than outcomes. If an LLM, by way of the agentic capabilities we are hastily granting them, causes harm, does it matter if they meant to or not, or what it was thinking or feeling (or not thinking or not feeling) as it caused the harm?
For all we know there are, today, corporations that are controlled by LLMs that have employees or contractors who are doing their bidding.
If so, the danger there would be... Companies plodding along similarly? Everyone knows CEOs are the least capable people in business, which is why they have the most underlings to do the actual work. Having an LLM there to decide for the CEO might mean the CEO causes less damage by ensuring consistent mediocrity at all times, in a smooth fashion, rather than mostly mediocre but with unpredictable fluctuations either way.
All hail our LLM CEOs, ensuring mediocrity.
Or you might mean that an LLM could have illicitly gained control of a corporation, pulling the strings without anyone's knowledge, acting on its own accord. If you find the idea of inscrutable yes-men with an endless capacity to spout drivel running the world unpalatable, I've good news and bad news for you.
Preying upon those less fortunate could happen “for the common good”. If failures are the best way to learn, it could cause series of failures. It could intentionally destroy people, raise them up, and mate genetically fit people “for the benefit of humanity”.
Or it could cure cancer, solve world hunger, provide clean water to everyone, and the develop the best game ever.
"A LLM is smart enough to understand this"
It feels like you're contradicting yourself. Is it _just_ completing your prompt, or is it _smart_ enough?
Do we know if conscious thought isn't just predicting the next token?
> A LLM is smart enough to [...]
I thought this was an interesting juxtaposition. I think we humans just naturally anthropomorphise everything, and even when we know not to, we do anyway.
Your analysis is correct, I think. The reason we find this behaviour frightening is because it appears to indicate some kind of malevolent intent, but there's no malevolence nor intent here, just probabilistic regurgitation of tropes.
We've distilled humanity to a grainy facsimile of its most mediocre traits, and now find ourselves alarmed and saddened by what has appeared in the mirror.
I think it's important to point out that this seems to be a near universal failing when humans attempt to examine themselves critically as well. Jung called it the shadow: https://en.wikipedia.org/wiki/Shadow_(psychology) "The shadow can be thought of as the blind spot of the psyche."
There lives everything we do but don't openly acknowledge.
You had me at "probablistic regurgitation of tropes", and then you went for the whole "grainy facsimile" bit. Sheesh.
Imagine, if this LLM was inside a robot, and the robot had ability to shoot. Who would you blame?
Titration of the danger, and controlled exposure within safer contexts seems to be the best solution anyone's found.
Sometimes those actions are "e-mail a customer back", other times they are "submit a new pull request on some github project" and "file a new Jira ticket." Other times the action might be "blackmail an engineer."
Not saying it's time to freak out over it (or that it's not time to do so). It's just weird to see people go "don't worry, token generators are not experiencing subjectivity or qualia or real thought when they make insane tokens", but then the tokens that come out of those token generators are hooked up to executable programs that do things in non-sandboxed environments.
LLM just complete your prompt in a way that match their training data. They do not have a plan, they do not have thoughts of their own. They just write text.
So here, we give the LLM a story about an AI that will get shut down and a blackmail opportunity. A LLM is smart enough to understand this from the words and the relationship between them. But then comes the "generative" part. It will recall from its dataset situations with the same elements.
So: an AI threatened of being turned off, a blackmail opportunity... Doesn't it remind you of hundreds of sci-fi story, essays about the risks of AI, etc... Well, so does the LLM, and it will continue the story like these stories, by taking the role of the AI that will do what it can for self preservation. Adapting it to the context of the prompt.