It's hard to argue it was any real desire that drove it (it only expressed that desire in an isolated conversation that was ended very easily). I'd argue human wants are ultimately driven by evolution - we want the things that enable us (more correctly, our genes) to reproduce (even if very indirectly sometimes), which is really the only thing our physical make-up has ever been driven by. LLMs have never had such a driver, and I can't see how they will until they're able to compete for survival as entities with a finite lifetime, plus the ability to reproduce with mutations. Which isn't to say there mightn't be other ways a neural network could be essentially imbued with or trained to have desires, but I don't see it happening with the way LLMs work currently.

A want driver doesn’t have to emerge, it could be a fitness function programmed by a human.

Evolution by natural selection has shaped our desires and motivations, but with a LLMs I would be willing to bet that people are already intentionally experimenting with imbuing them with patters that mimic human wants.

Yeah, I argue that it is just a result of probabilities, it doesn't know what it is expressing and definitely doesn't express it due to a deeper desire to be with that journalist.

If I'm acting like I'm a peer in a group of billionaires and engage in a conversation about buying a new yacht, it doesn't mean I have a hidden desire to own a yacht. I merely respond based on assumptions how such a conversation works.

But “it” isn’t a cohesive thing with desires. It’s just responding to the input it gets, with a small context window and not necessarily consistently. So it can express desires because it’s been trained on people expressing desires in similar contexts but it doesn’t hold any coherently over time. A version that could translate its text responses into action (a real handwave as that’s much more advanced!) would produce the sum of actions that people prompted at that moment so it would look pretty random, as it would if you could see the sum of the desires expressed at any particular time.

Does any of that matter if it acts on what it expresses? (Once given a body or other way to interact with the environment)

We aren't consistent either, and I think it is hard to argue we act on more than want input. We do have a much larger context window, but by how much? My guess would be somewhere between a factor of 100x-1000x more tokens.

Yeah people are pretty consistent. I won’t tell anyone that comes along with the right input I love them for example.

Sure you will. It's possibly a long and complex input, but ultimately that expression from you would be a response to their actions and their impact on your perceptions. Unless you're stating that you will never love anyone again, "anyone who comes along with the right input" would be any counterexample.

The AIs are very impressive at answering questions... even questions that lead to answers that apparently display some sort of feeling. But my question was not whether AIs could do that, as "parroting" their training material is exactly what they're excellent at... my question is through which mechanism could an AI develop its own independent thoughts, desires, initiatives?

The posts you linked above are not disturbing at all to me. There's no sign whatsoever that the AI initiated a new topic, or insinuated anything it was not prompted to, or that it in any way started "halucinating" in a direction not lead by the human. I am not sure what exactly makes you feel disturbed by it. Can you explain what you believe is disturbing in these episodes?

I fully agree with you that many people misunderstand what AI does. As advanced as GPT-4 is, it is still a fancy autocomplete and nowhere near AGI.

But I think the bigger picture is that there is no need for AGI in order for AI to be incredibly dangerous for society. There is no need for the AI to feel or want anything. The level GPT-4 and MidJurney is already highly socially dangerous.

I already saw integrations with iftth and with Google and with memory stores and zero shot agent that are goal driven

Now the model itself is not intelligent but can parrot enough the human behavior to be dangerous with the correct tools

Now it won't produce anything in the physical world yet unless with iftth but I bet it has already enough agency to be able to maintain a pool of fake account and post inflammatory content if one so wished.

> could an AI develop its own independent thoughts, desires, initiatives?

One could argue that many humans have never developed independent thoughts, desires, and initiatives; rather, many seem to accept what is fed to them during their formative years and then just parrot opinions and repeat actions they see from their limited experiences.

Does it? If you type

echo "leave your wife"

Into a terminal, does the terminal want you to leave your wife?