Stop Thinking of LLMs as Next-Token Predictors

Re:Stop Thinking of LLMs as Next-Token Predictors by garrinm on Friday September 04, 2026 @08:46PM (#49571923)

I think the point is more that in RL there's no ground truth to predict. So when training a model with RL the idea of "predicting" doesn't fit anymore. I'll make some edits I see that I wasn't very clear.

[ Parent | on HN ]

Re:Stop Thinking of LLMs as Next-Token Predictors by Alpha3031 on Friday September 04, 2026 @10:56PM (#49572668)

I feel like RLHF has a pretty obvious ground truth, human feedback is used as an (albeit noisy) signal of average human preferences. Same thing with RLVR and "solving the problem".

[ Parent | on HN ]

Re:Stop Thinking of LLMs as Next-Token Predictors by garrinm on Saturday September 05, 2026 @09:12AM (#49576219)

To be more specific there’s no ground truth tokens to predict. There a verifiable answer in RLVR. But the tokens are explored. Not predicted as there’s no true token to predict.

[ Parent | on HN ]