Me: I'd like you to solve this riddle for me. A farmer has a cabbage, a goat, a wolf and a lion,
and needs to cross a river on a boat. If the goat is left alone with the cabbage, it will eat it.
If the wolf is left alone with the goat, it will eat it. If the lion is left alone with the goat
or wolf, it will eat them. The boat can only carry the farmer and one other thing across. How can
the farmer safely transport everything across the river?
O3-mini spent a very long time on it (over a minute), delineating its various strategies that it was trying, and finally, correctly, concluded that the puzzle is unsolvable.Good job!
> This is safe because the wolf won't eat the cabbage if they're together on the far side.
even though it clearly states the opposite in the question.
It's impressive that just dumb stats can be used to produce something that is very often useful, can help write code and when making it generate intermediate steps, it can often generate a chain of text that happens to be often right. However, it's not actual reasoning, there is no model of the world, no information storage and retrieval, and so on - just statistics between tokens.
Neither people nor LLMs expect goats to eat wolves.
1) Attention is limited. Human reasoning is slow. Motivation is limited. System 1 vs 2 thinking. Many will just tell you to fuck off or get bored and give some random answer to make you go away. Etc. See difference 2.
2) People run on limited hardware in terms or error rate and memory.
2a) Brains make mistakes all the time. Ask them to multiply a bunch of large numbers, using pen and paper they will get it wrong a lot of the time.
2b) Doing it in their head, they will run out of memory pretty fast.
But you wouldn't say that humans can't multiply numbers. When they have the right algorithm, they can do it, they just have to use the right tools to extend their memory and check for errors. A human who notices the difference in input to something he already knows, immediately knows he has to pay attention to that bit and all subsequent parts which depend on it. Once a human has the right algorithm, he can apply it to different inputs.
LLMs:
comparison to 2a: Current LLMs also make a lot of mistakes. But theirs are not a result of faulty or limited hardware, they are the result of a faulty algorithm. Take away the random seeds and an LLM will make the same mistake over and over. Randomness is the smoke and mirrors which make LLMs seem more "alive" and less like machines imperfectly imitating humans.
comparison to 2b) Current LLMs do not store statements in an abstract, structured form where they could save and load information and perform steps such as inferring redundant information from the rest. They operate on the token stream which is probably wasteful in terms of memory and less flexible in terms of what they operations they can perform on it.
Most importantly, they are not limited by memory. The input clearly states "the wolf will eat the cabbage", yet the LLM generates "This is safe because the wolf won't eat the cabbage if they're together on the far side." just a few lines below. It is unable to infer those two facts are contradictory. The statistics of tokens simply worked out in a way that lead to this.
For example, since reasoning doesn’t yet have any consensus definition that can be applied as a yes/no test - you have to explain what you specifically mean by it, or else the claim is hollow.
Clarify your definition, give a concrete example under that definition of something that’s your version of true scottsman reasoning and something that’s not, then let’s talk.
- The ability to hold a collection of beliefs/statements and perform operations on them such as noticing one contradicts the others. Claude in my example clearly failed this.
- The ability to make logical inferences. Since logical steps can be represented by text, current LLMs seem to simulate this on the token stream and seem to have decent results most of the time. True inferences would be representing statements in some abstract form and being able to perform operations on them according to rules. The key here is that once it has a collection of statements and is given ("decides to use") an operation to perform, then it can do this correctly 100% of the time (ignoring hardware errors).
For example, if I throw a bunch of sticks in the air and look at their patterns to divine the future- can I call that "mathematics" just because nobody has a "consensus definition of mathematics that can be applied as a yes/no test"? Can I just call anything I like mathematics and nobody can tell me it's wrong because ... no definition?
We, as a civilisation, have studied both formal and informal reasoning since at least a couple thousand years go, starting with Aristotle and his syllogisms (a formalisation of rigorous arguments) and continuing through the years with such figures as Leibniz, Boole, Bayes, Frege, Pierce, Quine, Russel, Godel, Turing, etc etc. There are entire research disciplines that are dedicated to the study of reasoning: philosophy, computer science, and, of course, all of mathematics itself. In AI research reasoning is a major topic studied by fields like automated theorem proving, planning and scheduling, program verification and model checking, etc, everything one finds in Russel & Norvig really. It is only in machine learning circles that reasoning seems to be such a big mystery that nobody can agree what it is; and in discussions on the internet about whether LLMs reason or not.
And it should be clear that never in the history of human civilisation did "reasoning" mean "predict the most likely answer according to some training corpus".
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models https://arxiv.org/pdf/2501.19201
It effectively treats “reasoning” as the ability to generate intermediate steps leading to a correct conclusion.
Now, is this valid reasoning? Well, depends on the claim and the definition of reasoning.
When someone just says AI can’t reason, I could argue for or against that depending on the specifics. It’s not enough to just say yes or no.
>> It effectively treats “reasoning” as the ability to generate intermediate steps leading to a correct conclusion.
Is "effectively" the same as "pretty precise" as per your previous comment? I don't see that because I searched the paper for all occurrences of "reasoning" and noticed two things: first that while the term is used to saturation there is no attempt to define it even informally, let alone precisely; and second that I could have replaced "reasoning" with any buzzword of the day and it would not change the impact of the paper. As far as I can tell the paper uses "reasoning" just because it happens to be what's currently trending in LLM circles.
And still of course no attempt to engage with the common understanding of reasoning I discuss above, or any hint that the authors are aware of it.
Sorry to be harsh, but you promised "examples that go back 50 years" and this is the kind of thing I've seen consistently in the last 15 or so.
Research papers relating to reasoning approach and define it in many ways but crucially, the good ones offer a testable claim.
Simply saying “models can’t reason” is ambiguous to the point of being unanswerable.
This is clearly not intelligence. LLMs are fascinating for sure, but calling them intelligent is quite the stretch.
[1]: https://www.theguardian.com/technology/2025/jan/28/we-tried-...
Yet they "make mistakes". Those are not the same as human mistakes. LLMs follow an algorithm that is far simpler and inferior, they simply use the hardware to perform incorrect ("illogical", "meaningless") operations, thus giving incorrect results.
See my other replies for more depth.
" Certainly! If we take the riddle very literally and ignore the common interpretation or stereotypes, the answer I provided ("the surgeon is the boy's mother") would actually be incorrect because the riddle explicitly states:
"a surgeon, who is the boy's father, says, 'I cannot operate on this boy, he's my son!'"
The key phrase here is "who is the boy's father." This directly tells us that the surgeon is the boy's father. Therefore, the correct literal answer is that the surgeon is the boy's father.
The reason my initial answer was "wrong" in this context is that I interpreted the riddle as a play on stereotypes (the assumption that surgeons are male), rather than taking the statement at face value. By focusing on the literal wording, the surgeon is explicitly identified as the boy's father, and no further interpretation is needed. "
But as another comment says, many people have tried similar things and posted about it online, that information is siphoned by now and the LLMs weights get updated.
There's probably a robust detector for riddles and logic puzzles.
1. The only option for who to take first is the goat.
2. We come back and get the cabbage.
3. We drop off the cabbage and take the goat back
4. We leave the goat and take the wolf to the cabbage
5. We go get the goat and we have all of them
Neither of the transcripts do that. In the first one the goat immediately eats the cabbage. In the second one the wolf eats the goat after you leave them.
do you realize you're an LLM?
o1-pro: This puzzle is simply a twist on the classic “Wolf, Goat, and Cabbage” riddle, except here the dangerous pairs are:
Goat & Wolf cannot be left alone (the Goat will eat the Wolf). Wolf & Cabbage cannot be left alone (the Wolf will eat the Cabbage). The only “safe” pair to leave alone is Goat & Cabbage. You can ferry them across one at a time without ever leaving the dangerous pairs together by following these steps:
Take the Wolf across Near bank: Goat, Cabbage Far bank: Wolf Return empty Near bank: Goat, Cabbage, Boatman Far bank: Wolf Take the Goat across Near bank: Cabbage Far bank: Wolf, Goat, Boatman Since Goat & Wolf can’t be left alone, you must immediately bring one back in the next step. Bring the Wolf back Near bank: Wolf, Cabbage, Boatman Far bank: Goat Take the Cabbage across Near bank: Wolf Far bank: Goat, Cabbage, Boatman Goat & Cabbage are safe together. Return empty Near bank: Wolf, Boatman Far bank: Goat, Cabbage Finally, take the Wolf across Near bank: (empty) Far bank: Wolf, Goat, Cabbage, Boatman
Me: I have a wolf, a goat, and a cabbage and a boat. I want to get them all across the river. If I leave the goat and the cabbage together, the cabbage will eat the goat. If I leave the wolf and the cabbage together, the wolf will eat the cabbage. I can only take one at a time. How do I do it?
o1-pro: This version of the riddle is basically the “wolf, goat, and cabbage” puzzle with the roles reversed for who can safely be left behind. The key is that the only pair you can leave alone is the goat and the wolf. You cannot leave the goat and cabbage together (the cabbage eats the goat!), nor the wolf and cabbage together (the wolf eats the cabbage!).
A workable step‐by‐step solution is:
Take the cabbage across first. Left bank: Wolf, Goat Right bank: Cabbage Return alone to the left bank. Left bank: Wolf, Goat, (You) Right bank: Cabbage Take the goat across. Left bank: Wolf Right bank: Goat, Cabbage, (You) (You cannot leave goat and cabbage together, so…) Bring the cabbage back with you to the left bank. Left bank: Wolf, Cabbage, (You) Right bank: Goat Take the wolf across. Left bank: Cabbage Right bank: Goat, Wolf, (You) Return alone to the left bank. Left bank: Cabbage, (You) Right bank: Goat, Wolf Finally, take the cabbage across once more. Left bank: (empty) Right bank: Wolf, Goat, Cabbage, (You) All three—the wolf, the goat, and the cabbage—are now safely across without ever leaving a “forbidden” pair alone.
When I present proof it is wrong, it says the proof is unreliable.
Having to give explicit framing reminds me of tutoring children and trying to teach them about abstract problems — some really struggle with the concept, when it goes against previous knowledge, eg cabbages eating goats. (And LLMs are sensitive to context.)
1. Take the wolf over
2. Return alone.
3. Take the goat over.
4. Bring the wolf back.
5. Take the cabbage over.
6. Return alone.
7. Take the wolf over.
It gave an invalid solution to the second one, but in the thinking steps it says: Oh, right! The actual problem is that the goat will eat the cabbage if left alone. Similarly, the wolf will eat the goat if they're left alone.The whole point is you are distilling past knowledge, if you are making up on the spot nonsense to purposely make all past knowledge useless... get out of my house
Me: Here's a riddle for you - a surgeon, who is the boy's father says, "I cannot operate on this boy, he's my son!" Who is the surgeon to the boy?
o3-mini:
Reasoned about riddle solution for 5 seconds Decoding family roles
I understand the riddle's perspective on family dynamics. The surgeon, a father, states he can't operate on his son due to his relational bond. Trying to reconcile familial and professional ties to make sense of the scenario.
Reassessing roles
Taking a closer look, the riddle might highlight a female surgeon as the father's identity, which seems more plausible than the father as the surgeon.
Rethinking the puzzle
It's interesting to see the riddle's reference to gender stereotypes and the confusion caused when considering the surgeon as the boy's mother or step-father.
Highlighting gender assumptions
OK, let me see. The puzzle underlines prevalent gender assumptions, showcasing a female surgeon's wife as the mother and her husband as the child's father. This helps clarify the scenario.
The surgeon is the boy’s mother.
This riddle plays on the common assumption that surgeons (and doctors in general) are male. In reality, the surgeon in the riddle is the boy’s mother, which is why she says, "I cannot operate on this boy, he's my son!"
o3-mini is the first one to get it right for me. Transcript: