Formal reasoning is defined, informal reasoning very much isn't.

At the end of the day most people would agree that if something is able to solve a problem without a lookup table / memorisation that it used reasoning to reach the answer. You are really just splitting hairs here.

What do "most" people thinking about LLMs, then?

The "hair-splitting" underlies the whole GenAI debate.

We have widely used benchmarks for reasoning. And no no it does not, you need get off HN.

The difference between thinking and reasoning is that I can "think" that Elvis is still alive, Jewish space lasers are responsible for California wildfires, and Trump was re-elected president in 2020, but I cannot "reason" myself into those positions.

It ties into another aspect of these perennial threads, where it is somehow OK for humans to engage in deluded or hallucinatory thought, but when an AI model does it, it proves they don't "think."