Large Language Model Reasoning Failures

Re:Large Language Model Reasoning Failures by sergiomattei on Saturday February 21, 2026 @05:50AM (#47099504)

Papers like these are much needed bucket of ice water. We antropomorphize these systems too much.

Skimming through conclusions and results, the authors conclude that LLMs exhibit failures across many axes we'd find to be demonstrative of AGI. Moral reasoning, simple things like counting that a toddler can do, etc. They're just not human and you can reasonably hypothesize most of these failures stem from their nature as next-token predictors that happen to usually do what you want.

So. If you've got OpenClaw running and thinking you've got Jarvis from Iron Man, this is probably a good read to ground yourself.

Note there's a GitHub repo compiling these failures from the authors: https://github.com/Peiyang-Song/Awesome-LLM-Reasoning-Failur...

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by vagrantstreet on Saturday February 21, 2026 @06:57AM (#47099948)

Isn't it strange that we expect them to act like humans even though after a model was trained it remains static? How is this supposed to be even close to "human like" anyway

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by mettamage on Saturday February 21, 2026 @07:36AM (#47100220)

> Isn't it strange that we expect them to act like humans even though after a model was trained it remains static?

An LLM is more akin to interacting with a quirky human that has anterograde amnesia because it can't form long-term memories anymore, it can only follow you in a long-ish conversation.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by LiamPowell on Saturday February 21, 2026 @07:14AM (#47100068)

If we could reset a human to a prior state after a conversation then would conversations with them not still be "human like"?

I'm not arguing that LLMs are human here, just that your reasoning doesn't make sense.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by hackinthebochs on Saturday February 21, 2026 @07:47AM (#47100315)

Henry Molaison was exactly this.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by alansaber on Saturday February 21, 2026 @09:43AM (#47101291)

I mean you can continue to evolve the model weights but the performance would suck so we don't do it. Models are built to an optimal state for a general set of benchmarks, and weights are frozen in that state.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by lostmsu on Saturday February 21, 2026 @07:40AM (#47100257)

https://en.wikipedia.org/wiki/List_of_cognitive_biases

Specifically, the idea that LLMs fail to solve some tasks correctly due to fundamental limitations where humans also fail periodically well may be an instance of the fundamental attribution error.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by otabdeveloper4 on Saturday February 21, 2026 @08:31AM (#47100677)

> We antropomorphize these systems too much.

They're sold as AGI by the cloud providers and the whole stock market scam will collapse if normies are allowed to peek behind the curtain.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by alansaber on Saturday February 21, 2026 @09:44AM (#47101299)

The stock market being built on conjecture? Surely not sir.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by simianwords on Saturday February 21, 2026 @09:08AM (#47100996)

Most of the claims are likely falsified using current models. I wouldn’t take many of them seriously.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by jibal on Saturday February 21, 2026 @09:58AM (#47101429)

I wouldn't take baseless "likely" claims or the people who make them seriously.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by simianwords on Saturday February 21, 2026 @12:24PM (#47102750)

I falsified it on another thread

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by throw310822 on Saturday February 21, 2026 @09:15AM (#47101041)

> conclude that LLMs exhibit failures across many axes we'd find to be demonstrative of AGI.

Which LLMs? There's tons of them and more powerful ones appear every month.

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by alansaber on Saturday February 21, 2026 @09:40AM (#47101262)

True but the fundamental architecture tends not to be radically different, it's more about the training/RL regime

[ Parent | on HN ]

Re:Large Language Model Reasoning Failures by throw310822 on Saturday February 21, 2026 @09:54AM (#47101394)

But the point is that to even start to claim that a limitation holds for all LLMs you can't use empirical results that have been demonstrated only for a few old models. You either have a theoretical proof, or you have empirical results that hold for all existing models, including the latest ones.

[ Parent | on HN ]