It goes up at least until LLMs match humans - ie until an LLM can write Windows

Slightly tangential, but I want an LLM which can debug windows.

Debug And locally fix security holes.

Or sell the found exploit to the friendly nation state actor to pay for its compute behind your back

I want the LLM to decide not to do anything, or write a new OS.

Whenever I prompt: "Do not do anything"

It always does <something>.

> Whenever I prompt: "Do not do anything" It always does <something>.

Yep. A lot of times, the responses I get remind me of Simone in Ferris Bueller's Day Off: https://www.youtube.com/watch?v=swBtLPWeKbU

If you end up making a new model, please teach it that less is more and call it "LAIconic".

Do not think of a pink elephant. Were you able to do so?

Isn't the question closer to "/when/ does line stop going up?"

That is a matter of hope.

If line keeps going up, line does catastrophic or potentially apocalyptic harm, given our current circumstances.