Re:Reasoning models don't always say what they think
by Robin_Message on Thursday April 03, 2025 @03:44PM
(#43574457)
The weights are aware of the end goal etc. But the model does not have access to these weights in a meaningful way in the chain of thought model.
So the model thinks ahead but cannot reason about it's own thinking in a real way. It is rationalizing, not rational.
Re:Reasoning models don't always say what they think
by Zee2 on Thursday April 03, 2025 @05:01PM
(#43575356)
I too have no access to the patterns of my neuron's firing - I can only think and observe as the result of them.
Re:Reasoning models don't always say what they think
by senordevnyc on Thursday April 03, 2025 @08:48PM
(#43577203)
So the model thinks ahead but cannot reason about its own thinking in a real way. It is rationalizing, not rational.
My understanding is that we can’t either. We essentially make up post-hoc stories to explain our thoughts and decisions.
This article counters a significant portion of what you put forward.
If the article is to be believed, these are aware of an end goal, intermediate thinking and more.
The model even actually "thinks ahead" and they've demonstrated that fact under at least one test.