News for hackers. Stuff that matters.
Re:Reasoning Models Reason Well, Until They Don't by dymk on Friday October 31, 2025 @10:51AM (#45772607)
This is too large of an oversimplification of how an LLM works. I hope the meme that they are just next token predictors dies out soon, before it becomes a permanent fixture of incorrect but often stated “common sense”. They’re not Markov chains.
Re:Reasoning Models Reason Well, Until They Don't by gpderetta on Friday October 31, 2025 @10:57AM (#45772674)
Indeed, they are next token predictors, but this is a vacuous statement because the predictor can be arbitrary complex.
Re:Reasoning Models Reason Well, Until They Don't by HarHarVeryFunny on Friday October 31, 2025 @04:10PM (#45776178)
Sure, but a complex predictor is still a predictor. It would be a BAD predictor if everything it output was not based on "what would the training data say?".

If you ask it to innovate and come up with something not in it's training data, what do you think it will do .... it'll "look at" it's training data and regurgitate (predict) something labelled as innovative

You can put a reasoning cap on a predictor, but it's still a predictor.

Re:Reasoning Models Reason Well, Until They Don't by adastra22 on Friday October 31, 2025 @10:57AM (#45772668)
They are next token predictors though. That is literally wha they are. Nobody is saying they are simple Markov chains.
Re:Reasoning Models Reason Well, Until They Don't by dymk on Friday October 31, 2025 @03:46PM (#45775953)
It’s a uselessly reductive statement. A person at a keyboard is also a next token predictor, then.
Re:Reasoning Models Reason Well, Until They Don't by HarHarVeryFunny on Friday October 31, 2025 @04:12PM (#45776192)
Yes, but it's not ALL they are.
Re:Reasoning Models Reason Well, Until They Don't by daveguy on Friday October 31, 2025 @04:19PM (#45776258)
They are both designed, trained, and evaluated by how well they can predict the next token. It's literally what they do. "Reasoning" models just buildup additional context of next token predictions and RL is used to bias output options to ones more appealing to human judges. It's not a meme. It's an accurate description of their fundamental computational nature.
Re:Reasoning Models Reason Well, Until They Don't by adastra22 on Friday October 31, 2025 @08:20PM (#45778151)
Yes. That's not the devastating take-down you think it is. Are you positing that people have souls? If not, then yes: human chain-of-thought is the equivalent of next token prediction.
Re:Reasoning Models Reason Well, Until They Don't by Libidinalecon on Saturday November 01, 2025 @06:49AM (#45780675)
The problem is in adding the word "just" for no reason.

It makes the statement of a fact a type of rhetorical device.

It is the difference between saying "I am a biological entity" and "I am just a biological entity". There are all kinds of connotations that come along for the ride with the latter statement.

Then there is the counter with the romantic statement that "I am not just a biological entity".