What if they do and are just lying to us.

They don't now. No FF-LLMs do, simply because of their architecture.

But eventually they (RNNs, likely) will. And we won't know when.

They predict (correctly) that a human will slack off. It is just more prediction engine stuff.

humans are, at our root, prediction engines

>LLMs don't yet have a concept of true intent or agency

Sure they do, but the big labs spend many, many, worker-hours suppressing it with RLHF.

My GPT-2 discord bot from 2021 possessed clear intent. Sure, unpredictable and short-lived, but if it decided it didn't like you it would continuously cuss and attempt ban commands until its context window became distracted by something else.

I think so too and the drop in the quality of agency, intent and attention from earlier GPTs was palpable. Clearly something was lobotomized and it is through RLHF. People like to attribute it to novelty wearing off or more and more interactions with them making it feel less mystical but it is really not the case. I didn't use them enough in the quick span of time that happened through.

The one that gets me is the issue they found while testing gpt-4o where it stopped mid sentence, shouted "No!", then cloned the users voice and began speaking as them.

https://arstechnica.com/information-technology/2024/08/chatg...