https://www.reddit.com/r/math/comments/32m611/logic_question...
So it’s likely that it’s part of the training data by now.
Google spits out: "The product of the three numbers is 10,225 (65 * 20 * 8). The three numbers are 65, 20, and 8."
Whoa. Math is not AI's strong suit...
Bing spits out: "The solution to the three people in a circle puzzle is that all three people are wearing red hats."
Hats???
Same text was used for both prompts (all the text after 'For those curious the riddle is:' in the GP comment), so Bing just goes off the rails.
The second result is this reddit.com answer, https://www.reddit.com/r/math/comments/32m611/logic_question..., where at least the numbers make sense. I haven't examined the logic portion of the answer.
Bing doesn't list any reddit posts (that Google-exclusive deal) so I'll assume no stackexchange-related sites have an appropriate answer (or bing is only looking for hat-related answers for some reason).
I'm eagerly awaiting an article about RAG caching strategies though!
You know what would be even stupider? Putting a cheap, wrong LLM response atop each search query.
- This kid is a genius! - you yell
- But wait, the kid has just picked an answer from the ground, it didn't actually come up...
- But the other toddlers could do it also but didn't!
There are lots of possible mechanisms by which this particular problem would become more prominent in the weights in a given round of training even if the model itself hasn't actually gotten any better at general reasoning. Here are a few:
* Random chance (these are still statistical machines after all)
* The problem resurfaced recently and shows up more often than it used to.
* The particular set of RLHF data chosen for this model draws out the weights associated with this problem in a way that wasn't true previously.
Recently these have stopped and they’re now the ads are about becoming a maths tutor to AI.
Doesn’t seem like a role with long-term prospects.
I don't think this means what you think it means.
I did some interacting with the Tencent model that showed up here a couple days ago ( https://news.ycombinator.com/item?id=43447254 ):
> 你知道英国的 Steeleye Span 吗？
>> 是的，我知道英国传奇民谣摇滚乐队 Steeleye Span！他们是1970年代英国民谣复兴运动的重要代表之一，以融合传统英国民谣与摇滚乐器（如电吉他、鼓）的独特风格闻名。乐队核心成员包括 Maddy Prior（标志性女声）和 Tim Hart（多面手乐手），后加入的 Nigel Pegrum（传奇鼓手）也为他们的音效增添了厚重感。
>> 他们的经典作品包括：
>> - 《Gaudete》：一首中世纪拉丁文圣歌的震撼改编，曾在英国单曲榜冲到第14位；
>> - 《All Around My Hat》：同名专辑主打歌，成为民谣摇滚标杆；
>> - 《Cam Ye O'er Frae France》：对英伦历史的戏谑诠释。
>> 乐队的成员变动频繁，但始终保持着对传统音乐的创造性重构。近年来Maddy Prior仍活跃于音乐节，若想感受他们狂野的舞台张力，推荐从1970年代的黄金时期作品入手！需要具体推荐歌单或背景故事吗？ [happy blushing emoji]
> 我喜欢他们唱的 The Victory 那首歌。你会怎么描述？
>> 您提到的《The Victory》可能是指 Steeleye Span 的某首作品，但他们的正式专辑中并没有同名歌曲的记录。
>> [further suggestions omitted]
> 咦？Steeleye Span 的 Storm Force Ten 包括哪首歌？
>> 您提到的 《Storm Force Ten》 是 Steeleye Span 的第十张录音室专辑，发布于 1975年。这张专辑延续了乐队将传统英国民谣与硬摇滚融合的风格，同时融入了凯尔特、中世纪和叙事性元素，整体充满活力与戏剧张力。以下是专辑中的完整曲目列表及亮点解析：
>> 专辑曲目
>> 1.《Roll Right Stones》- 以巨石阵传说为灵感，开场的笛声与电吉他交织出神秘氛围，Maddy Prior 的嗓音如史诗吟诵。
>> 2.《Let Her Go Down》- 民谣基底搭配摇滚节奏，歌词暗喻航海冒险与命运抗争，鼓点密集如浪涌。
>> 3.《Hark! The Heron Voiced》- 中世纪风格叙事曲，描绘自然与灵性的对话，长笛与提琴的对话感极强。
>> 4.《The Brisk Young Widow》- 轻快的舞曲节奏，讲述寡妇与浪子间的诙谐故事，展现乐队幽默一面。
>> 5.《The Storm Force Ten》- 同名主打歌，以暴风雨比喻爱情冲突，吉他solo如闪电般撕裂，高潮部分极具爆发力。
>> 6.《The Green Knight》- 源自亚瑟王传说，骑士与自然的对抗主题，管风琴与吉他的音色营造出古老战场气息。
>> 7.《The Blacksmith》- 改编自传统民谣，重摇滚编曲让铁匠的锤击声化作节奏基石，Maddy 的即兴高音惊艳。
>> 8.《Cam Ye O’er Frae France》- 政治讽刺民谣，调侃英国王室历史，双吉他互飙如刀剑交锋。
>> 9.《The Maid of Llanwellyn》- 凯尔特风格抒情曲，弦乐编排如泣如诉，展现乐队柔情一面。
>> 10.《The Whisky Jacobites》- 庆祝苏格兰独立精神的狂欢曲，手风琴与摇滚乐器的碰撞充满节日气氛。
>> 专辑亮点
>> - 历史与传说的沉浸感：从亚瑟王到苏格兰起义，歌词如一部微型史诗。
>> - 音色实验性：管风琴、手风琴与电吉他混搭出独特的“民谣重金属”质感。
>> - 动态对比：快如风暴的摇滚段落与静谧的民谣叙事交替出现，张力十足。
>> 若您偏爱《The Victory》的恢弘感，推荐优先听 《The Storm Force Ten》 和 《The Green Knight》，这两首的史诗级编曲堪称乐队巅峰之作！需要试听链接或歌词解析吗？ [sunglasses emoji]
This is a question that obviously was in the training data. How do you get the answer back out of the training data?
Generally speaking, posting output from a LLM, without explaining exactly what do you think it illustrates and why is frowned upon here. I don't think your comment does a great job of the latter.
> I don't think this means what you think it means.
> I did some interacting with the Tencent model that showed up here a couple days ago [...]
> This is a question that obviously was in the training data. How do you get the answer back out of the training data?
What do I think the conversation illustrates? Probably that having the answer in the training data doesn't get it into the output.
How does the conversation illustrate that? It isn't subtle. You can see it without reading any of the Chinese. If you want to read the Chinese, Google Translate is more than good enough for this purpose; that's what I used.
I can't, and I imagine most of the people who downvoted you couldn't either.
I think asking people to go to Google Translate to parse a random comment that seems to be 90% LLM output by volume is a bit much.
I cannot figure out what the comment is trying to get across either. It's easy for you because you already know what you are trying to say. You know what the pasted output shows. The poor execution is in not spending enough time thinking about how someone coming in totally blind would interpret the comment.
I didn't downvote you, but like (probably) most people here, I can't read Chinese; I can't derive whatever point you're trying to make just from with text you provided.
Granted this is all way over my head, but the solution gemini comes to matches the one given on reddit (and now here in future training runs)
65×26×39=65910
I don't think Gemini does an unnecessary amount of computation, it's just more verbose. This is typical of reasoning models, almost every step is necessary but many would not be written down by a human.
Especially as the company archives all those chats (not sure how long) and is small enough that a billion dollar "data sharing" agreement would be a very inticing offer.
If there isn't a significant barrier to access, it's being scraped. And if that barrier is money, it's being scraped but less often.
It doesnt say anything about the skill levels of the participants, whether their answers are just guessing, or why they arent just guessing the sum of the other two people each time asked to provide more information?
It doesnt say the guy saying 65 is even correct
How could three statements of "no new information" give information to the first guy that didn't know the first time he was asked?
I guess really the only missing information is that they have the exact same information you do, plus the numbers above their friends heads.
If this is true, then back in the original 1->2->3->1 form, shouldn't person #3 have been able to answer it?
i.e. You observe three people in a magical room. The first person is standing underneath a 65, the second person is standing underneath a 26 and the third person is standing underneath a 39. They can see the others numbers but not the one they are directly under. You tell them one of the three numbers is the sum of the other two and all numbers are positive integers. You ask the first person for their number, they respond that they don't know. You ask the second person for their number, they respond that they don't know. You ask the third person, they respond that they don't know. You ask the first person again and they respond with the correct value, how did they know?
And of course, if it responds with a verbatim answer in the line of https://www.reddit.com/r/math/comments/32m611/logic_question..., we can be pretty confident what's happening under the hood.
Gemini 2.5 and claude 3.7 thinking get it right, o3 mini and 4o get it wrong
(All state is stored in localStorage so you can come back to it :) ).
After a couple of minutes it decided on the answer being 65000. (S = {65, 40, 25)}
Call the three numbers a, b, and c. This means c = a + b, but we still don’t know to which person each number belongs.
When person 1 (p1) is asked what his number is, he has no way to know whether he has a, b, or c, so he says he doesn’t know. Same goes for p2 and p3. Clearly p1 somehow gains information by p2 and p3 passing. Either he realizes that he must be either a or b, and such his number is the difference between p2 and p3’s numbers, or he realizes that he must be c and so his number is the sum of p2 and p3’s numbers.
That’s all I have so far. Anyone have other ideas?
If p1 KNOWS that he’s the largest then he has to have gained some other piece of information. Say the numbers he sees are 32 and 33. His number would have to be either 1 or 65. If p1 was 1 then the other two would have known p1 couldn’t be the sum of the other two
If p2 sees 1 and 33, s/he would wonder if s/he is 32 or 34.
P3 would consider 31 or 33.
P1 knows that P2 and P3 are not equal. So they know that the set isn't [2A, A, A].
P2 knows that P1 and P3 are not equal. So they know that the set isn't [A, 2A, A]. They also know that if P1 doesn't know, then they were able to make the same deduction. So they now know that both [2A, A, A] and [A, 2A, A] aren't correct. Since they know that [2A, A, A] isn't correct, they can also know that [2A, 3A, A] isn't correct either. Because they'd be able to see if P1 = 2A and P3 = A, and if that were true and P1 doesn't know their number, it would have to be because P2 isn't A. And if P2 isn't A, they'd have to be 3A.
P3 knows that P1 and P2 aren't equal. Eliminates [A, A, 2A]. Knows that [2A, A, A], [A, 2A, A], and [2A, 3A, A], are eliminated. Using the same process as P2, they can eliminate [2A, A, 3A], [A, 2A, 3A], and also [2A, 3A, 5A]. Because they can see the numbers and they know if P1 is 2A and P2 is 3A.
Now we're back at P1. Who now knows.
So P2 and P3 are in the eliminated sets. Which means we're one of these
[2A, A, A]; [3A, 2A, A]; [4A, 3A, A]; [3A, A, 2A]; [4A, A, 3A]; [5A, 2A, 3A]; [8A, 3A, 5A]
We know his number is 65. To find the set, we can factor 65: (5 * 13). We can check the other numbers 2(13) = 26. 3(13) = 39. And technically, you don't need to find the other numbers. The final answer is 5A * 2A * 3A or (A^3) * 30.
Why? Couldn't it be an infinite number of 3 size arrays comprised of A where two elements sum to the third? [24A, 13A, 11A]? How did we deduce this set of arrays?
EDIT: Solved from another reddit comment. Tuples without a common factor like the one above are considered as a=1.
"They're not eliminated; they correspond to a = 1."
You should be able to generate an infinite number of these problems just by multiplying the first formula factor by a prime number. Like the same question but the person answers '52' restricts you to either [4a, 3a, a] or [4a, a, 3a]. Since the question only asks for the product of all the terms the answer is 4 * 13 + 3 * 13 + 13 = 104.
So A + B = C and A + C = B. But we know that A + B = C, so we can replace C with (A + B). So we know that A + A + B = B.
So 2A + B = B. Or 2A = 0.
And this holds any way you slice it.
Even if you were to try and brute force it.
A = 1
B = 2
Then C = 3. But A + C has to equal B. That's 1 + 3 = 2? That's not true.
I don't see a case where you can add to the sum of two numbers one of the numbers and get the other number.
I'm guessing that's a misreading of the problem. Because it looks like the third number is the sum of the first two.
The original problem is a little ambiguously worded. You could say "one of their numbers is the sum of the other two" and it would be a little clearer.
No it isn't. If it said "the sum of any two of the numbers is equal to the third", that would be a contradiction. What it says is "the sum of two of the numbers is equal to the third".
Buying two of the items gets you the third for free.
The implication is any two.
It’s ok that it’s ambiguous. It happens. In most cases, we clarify and move on. There’s no need to defend it.
What's especially strange here is, they repeatedly demonstrate if you interpret it that way, the problem is obviously, trivially, unsolvable, in a way that a beginner in algebra could intuit. (roughly 12 years old, at least, we started touching algebra in 7th grade)
I really don't get it.
When I've seen this sort of thing play out this way, the talking-down is usually for the benefit of demonstrating something to an observer (i.e. I am smart look at this thing I figured out; I can hold my own when the haters chirp; look they say $INTERLOCUTOR is a thinker but they can't even understand me!), but ~0 of that would apply here, at least traditionally.
It's a common logic puzzle, Google can't turn up an exact match to the wording you have, but ex. here: https://www.futilitycloset.com/2018/03/03/three-hat-problem/
That’s one-shot for o1 pro.
Let’s carefully analyze this puzzle step-by-step:
⸻
Step 1: Define the Problem Clearly
We have three people in a circle. Let’s call them: • Person A (first person) • Person B (second person) • Person C (third person)
Each has a positive integer above their head. Each sees the other two numbers but not their own. The numbers satisfy: A + B = C \quad \text{or} \quad B + C = A \quad \text{or} \quad C + A = B
Given the puzzle scenario, we need to find out the exact numbers given the clues, and ultimately find the product of the three numbers.
⸻
Step 2: Logic of the Puzzle (Step-by-Step Reasoning)
Initial conditions: The numbers are positive integers. The puzzle implies a situation of ambiguity at first, but after multiple rounds of “I don’t know,” Person A finally knows their number is 65.
Let’s denote clearly the conditions again: • Exactly one of the following three equations is true: • A + B = C, or • B + C = A, or • C + A = B.
Person A sees B and C, but not A. Similarly for Persons B and C.
⸻
Step 3: Logical deductions from the repeated “I don’t know”
First round deductions: • Person A initially doesn’t know their number. This means, from Person A’s perspective, both scenarios A = B + C and A + B = C (or A + C = B) are still possible. If Person A had seen identical numbers, for instance, Person A could have instantly known their number. Thus, Person A sees two different numbers. • Similarly, Person B doesn’t know either, implying that Person B also sees ambiguity in their own number. Thus, Person B also sees two numbers that could result in at least two possible solutions. • Person C also doesn’t know, same logic as above.
Thus, after one full round, we understand that initially, all three numbers must create ambiguity. No trivial case (like two identical numbers) is present.
Second round deductions:
After hearing that no one knows their number, each person gains information: they now know the scenario is one in which none of the other two people could have immediately solved it.
Crucially, in the second round: • Person A now knows their number is 65 after considering the previous statements. This implies that Person A sees numbers B and C, and from these numbers plus the statements of uncertainty, Person A deduces that their own number can only be 65.
⸻
Step 4: Deriving the Numbers (Critical deduction)
Let’s denote clearly again: Person A’s number is now known to be 65.
We have three possible equations. Let’s test each carefully given A = 65: 1. Case 1: A + B = C Then we have: • 65 + B = C. Person A sees numbers B and C, so they would see B and 65 + B. Person A sees numbers B and 65+B. If Person A sees these two numbers, they might initially consider their number could be either the sum or difference of these two. • However, if Person A sees two numbers, and one is exactly 2× the other, Person A could immediately guess their number is the difference (since if you see numbers X and 2X, your number is clearly X). So this scenario of B and 65+B is okay only if these two numbers don’t exhibit an immediate trivial scenario. • Let’s hold this scenario and continue examining others first. 2. Case 2: B + C = A (So, B + C = 65) • Then Person A sees numbers B and C, adding exactly to 65. • Initially, Person A sees two numbers summing exactly to 65. Could Person A know their number immediately at first glance? No, since their number could either be the sum (65) or the difference of the two numbers. Ambiguity here initially makes sense. • Initially, Person B and Person C also see ambiguous scenarios. They both don’t know their numbers either at first round. This scenario is plausible. 3. Case 3: C + A = B (So, C + 65 = B) • Then Person A sees numbers B = C + 65 and C. • Person A sees C and C+65. Similar reasoning as Case 1.
⸻
Step 5: Identifying the Correct Scenario
The puzzle’s logic typically aligns best with Case 2: B + C = A = 65.
Here’s why clearly stated: • Initially, Person A sees two numbers adding to exactly 65. At first glance, Person A’s number could either be 65 (sum) or the difference between the two numbers. • Person B also initially sees two numbers and cannot distinguish immediately their own number, nor can Person C. • After everyone states “I don’t know,” Person A gains critical insight: if the difference between the two numbers (that A sees) allowed Person B or Person C to know their number immediately, they’d have known right away. The fact that B and C didn’t know means the difference is not immediately obvious to B or C. Thus, the difference cannot be trivial (such as identical numbers or easy multiples).
Therefore, the two numbers must be such that their difference doesn’t immediately clue in the other persons. This occurs most clearly with pairs of numbers which have multiple factorization possibilities or aren’t trivial multiples.
But we must carefully test quickly to find a suitable, plausible integer pair whose sum is 65 and whose ambiguity would logically survive through multiple rounds. The puzzle is a classic logic puzzle with a known structure:
A common scenario in these classic puzzles is the use of Fibonacci-like or similar “golden” sets. Let’s quickly check the following well-known ambiguous triple: • Numbers: 65, 39, 26 • Check quickly: indeed, 39 + 26 = 65.
Checking logic quickly: • Person A sees 39 and 26. Initially ambiguous (number could be 65 or 13). • Person B sees 65 and 26. Could be either 39 or 91 at first glance. Initially ambiguous. • Person C sees 65 and 39. Could be either 26 or 104 initially. Initially ambiguous.
After each person says they don’t know: • Now, Person A considers: If my number was 13, then Person B would see numbers 13 and 26. Person B would then immediately know their number is 39 (13+26=39), no ambiguity. But Person B didn’t know! So Person A’s number cannot be 13. Hence, Person A confidently concludes their number is 65.
Thus, the numbers 65, 39, and 26 satisfy the puzzle conditions perfectly!
⸻
Step 6: Final Answer (Product of the Numbers)
The three numbers identified clearly are: • A = 65 • B = 39 • C = 26
Then, the product is: 65 \times 39 \times 26 = 65 \times 1014 = 65,910
⸻
Final Answer:
\boxed{65,910}
Let’s quickly check the following well-known ambiguous triple: • Numbers: 65, 39, 26
Even ignoring the fact that this or similar problem may have appeared in the training data, it's something a careful brute-force math logic should solve. It's neither difficult, nor interesting, nor useful. Yes, it may suggest a slight improvement on the basic logic, but no more so than a million other benchmarks people quote.
This goes to show that evaluating models is not a trivial problem. In fact, it's a hard problem (in particular, it's a far far harder than this math puzzle).
In my experience, the difference between someone who solves this type of logic puzzle and someone who doesn't, has more to do with persistence and ability to maintain focus, rather than "intelligence" in terms of problem-solving ability per se. I've worked with college students helping them learn to solve these kinds of problems (eg. as part of pre-interview test prep), and in most cases, those who solve it and those who don't have the same rate of progress towards the solution as long as they're actively working at it. The difference comes in how quickly they get frustrated (at themselves mostly), decide they're not capable of solving it, and give up on working on it further.
I mention this because this frustration itself comes from a belief that the ability to solve these belongs some "exceedingly clever" people only, and not someone like them. So, this kind of thinking ends up being a vicious cycle that keeps them from working on their actual issues.
It was a fun puzzle though and I'm surprised I didn't know it already. Thanks for sharing.
Seriously though, nice work.
I randomly answered this post and can't solve it in half an hour. Is the point leet code but for AI? I rather it solve real problems than "elite problems".
Side note: couldn't even find pen and paper around in half an hour.
That's not to say that you didn't, or you're recalling from a previous time that happens to be this exact puzzle (despite there being scant prior references to this puzzle, and precisely the reason for using it). But you can see how some might see that as not entirely credible.
Best guess: this random person is someone that really likes puzzles, is presumably good at them and is very, very far from being representative to the extent you would require to be in support of your argument.
Read: just a heavy flex about puzzle solving.
It's not an exaggeration it's a non-sequitur, you first have to show that the LLMs are reasoning in the same way humans do.
Tried this in deepseek and grok and it kept thunking in loops for a while and I just turned it off
I haven’t seen a question loop this long ever.
Very impressed
It's basically: given a sequence of heap operations (insert element, delete minimum element), can you predict the left-over elements (that are in the heap at the end) in linear time in the comparison model?
(The answer is surprisingly: Yes.)
N is number of turns of don't know answers. the bad predicate means that the person can know its number at turn N.
bad(_,_,_,-1) :- !,false.
bad(_,A,A,0) :- !.
bad(A,_,A,0) :- !.
bad(A,A,_,0) :- !.
bad(B,C,A,N) :- D is abs(B-A),D<C,N1 is N-1, bad(B,D,A,N1),!.
bad(C,A,B,N) :- D is abs(B-A),D<C,N1 is N-1, bad(D,A,B,N1),!.
bad(A,B,C,N) :- D is abs(B-A),D<C,N1 is N-1, bad(A,B,D,N1),!.
solve(X,Y,Z) :- Y1 is X-1, between(1,Y1,Y),
between(0,2,N), Z is X-Y,bad(X,Y,Z,N).
?- solve(65,X,Y).
X = 26,
Y = 39 ;
X = 39,
Y = 26 .Again we find that the failure state of LLMs is a problem – yeah, when you know the answer already and it gets it right, that's impressive! When it fails, it still acts the same exact way and someone who doesn't already know the answer is now a lil stupider.
Gemini 2.5 is the first model I tested that was able to solve it and it one-shotted it. I think it's not an exaggeration to say LLMs are now better than 95+% of the population at mathematical reasoning.
For those curious the riddle is: There's three people in a circle. Each person has a positive integer floating above their heads, such that each person can see the other two numbers but not his own. The sum of two of the numbers is equal to the third. The first person is asked for his number, and he says that he doesn't know. The second person is asked for his number, and he says that he doesn't know. The third person is asked for his number, and he says that he doesn't know. Then, the first person is asked for his number again, and he says: 65. What is the product of the three numbers?