Re:GPT-6 Astra covertly underperforms on safety research
by pizza234 on Thursday November 05, 2026 @04:50PM
(#50384960)
It's not about complexity but architecture. Neural networks work in a fundamentally different way from imperative programs.
You can modify curl (a "tool") and predict how its behavior will change, with a high degree of certainty (bugs aside).
You can't modify a neural network in the same way. You can train it to behave in a certain way, there are no guarantees that it will consistently behave as intended.
More importantly, neural networks encode far more information and behavior than we can fully understand/identify - this is the fundamental reason why LLMs aren't just tools.
You can modify curl (a "tool") and predict how its behavior will change, with a high degree of certainty (bugs aside).
You can't modify a neural network in the same way. You can train it to behave in a certain way, there are no guarantees that it will consistently behave as intended.
More importantly, neural networks encode far more information and behavior than we can fully understand/identify - this is the fundamental reason why LLMs aren't just tools.