Stop Thinking of LLMs as Next-Token Predictors

Re:Stop Thinking of LLMs as Next-Token Predictors by danielmarkbruce on Saturday September 05, 2026 @05:47PM (#49581014)

Many people are in fact claiming the thing you are saying they are not - even if you are not. The reason they are claiming it is that it was true at one point, and most intro courses/blog posts/videos still describe them that way and then hand wave some "other stuff at the end". You can even see a comment here that refers to the gpt-2 paper. LLMs were trained to predict the next token, produced a distribution to do so, were scored against their prediction v the truth, and the weights updated so that the probability distribution made it more likely to predict the truth from that sample next time. They were, in every sense of the word, a next token predictor.

They are no longer that thing due to post training. They simply aren't making a prediction, and they aren't even optimized for the next token. If I give a distribution of the heights of the population, I'm not giving a prediction either. Distributions don't imply predictions.

Why the desperation to hang onto the word "prediction"?

[ Parent | on HN ]

Re:Stop Thinking of LLMs as Next-Token Predictors by angoragoats on Sunday September 06, 2026 @03:09AM (#49584026)

> Many people are in fact claiming the thing you are saying they are not - even if you are not.

My original comment said “no one here.” Please show me where someone in the comments here is claiming that.

> Why the desperation to hang onto the word "prediction"?

No desperation here. It’s just a word that conveniently describes (especially to laypeople) what’s going on, even if it may not be the most mathematically correct or rigorous word to describe what’s going on. I think you’re being needlessly pedantic.

Why the desperation to refute it?

[ Parent | on HN ]