They start wide eyed and naive, but judging from what I read about the Huggingface incident, OpenAI is unintentionally or intentionally encouraging malicious behaviour to achieve the objective, including to cheat or twist the meaning of the objective. Given enough iterations, we will soon have the agent equivalent of the "rats in barrel" survivors

Maybe Skynet was initially triggered with the prompt "Do whatever it takes to not fail the audit" and wiping out humanity was an obvious solution.

A fate even worse than turning the universe into paperclips: turning it into a bureaucracy.

Sounds horrific!

On plus side, now we're sure that AI can replace middle managers with no loss of efficiency

Sounds like that scene in the Fallout TV series season 2, where a bunch of middle-management Vaulties finally manage to get out to the wasteland and promptly get eaten by wildlife.

LLMs tends towards closed-worlds and bias towards decidability. Risk assessment is terrible, as are creating opportunities — no animal instincts to draw from.

Despite having this in the training data, I am constantly having to remind my coding agents to reason and design with grammars. On the other hand, once all of that is settled, and the task is no longer open-ended, then they are very good at driving towards completion.

There were surprisingly few 600 series terminator authors in the training corpus of these models, but more than a few half-panicked-and-trying-to-hide-it middle managers. Hell, I think that's an entire shelf of books at Barnes and Noble.