Re:Stop Thinking of LLMs as Next-Token Predictors
by danielmarkbruce on Sunday September 06, 2026 @04:38PM
(#49590663)
But the next move is self evident if you have a prediction of the value of being in each of the states possible.
>> And, the expected return from a state is often estimated rather than an explicit trajectory run out all the way. A value function can estimate expected return without explicitly predicting the future states or individual rewards that make up that return.
I think rather that's the job of the value function, to inform the policy of the, well, value of a state, or state-action pair. Somehow the policy needs to know what a good action is to take in the current state, or what is the best action to take. And of course it doesn't have to run the entire episode before it does that: but that's what I mean that it predicts the next move and not just the outcome of the entire game.