You'd think so, but both Google's AI Overview and Bing's CoPilot output wrong answers.

Google spits out: "The product of the three numbers is 10,225 (65 * 20 * 8). The three numbers are 65, 20, and 8."

Whoa. Math is not AI's strong suit...

Bing spits out: "The solution to the three people in a circle puzzle is that all three people are wearing red hats."

Hats???

Same text was used for both prompts (all the text after 'For those curious the riddle is:' in the GP comment), so Bing just goes off the rails.

That's a non-sequitur, they would be stupid to run ab expensive _L_LM for every search query. This post is not about Google Search being replaced by Gemini 2.5 and/or a chatbot.

Google placed its "AI overview" answer at the top of the page.

The second result is this reddit.com answer, https://www.reddit.com/r/math/comments/32m611/logic_question..., where at least the numbers make sense. I haven't examined the logic portion of the answer.

Bing doesn't list any reddit posts (that Google-exclusive deal) so I'll assume no stackexchange-related sites have an appropriate answer (or bing is only looking for hat-related answers for some reason).

I might have been phrasing poorly. With _L_ (or L as intended), I meant their state-of-the-art model, which I presume Gemini 2.5 is (didn't come around to TFA yet). Not sure if this question is just about model size.

I'm eagerly awaiting an article about RAG caching strategies though!

Yes, putting an expensive LLM response atop each search query would be quite stupid.

You know what would be even stupider? Putting a cheap, wrong LLM response atop each search query.

The riddle has a different variants with hats https://erdos.sdslabs.co/problems/5

There's 3 toddlers on the floor. You ask them a hard mathematical question. One of the toddlers plays around pieces of paper on the ground and happens to raise one that has the right answer written on it.

- This kid is a genius! - you yell

- But wait, the kid has just picked an answer from the ground, it didn't actually come up...

- But the other toddlers could do it also but didn't!

Other models aren't able to solve it so there's something else happening besides it being in the training data. You can also vary the problem and give it a number like 85 instead of 65 and Gemini is still able to properly reason through the problem

I'm sure you're right that it's more than just it being in the training data, but that it's in the training data means that you can't draw any conclusions about general mathematical ability using just this as a benchmark, even if you substitute numbers.

There are lots of possible mechanisms by which this particular problem would become more prominent in the weights in a given round of training even if the model itself hasn't actually gotten any better at general reasoning. Here are a few:

* Random chance (these are still statistical machines after all)

* The problem resurfaced recently and shows up more often than it used to.

* The particular set of RLHF data chosen for this model draws out the weights associated with this problem in a way that wasn't true previously.

Google Gemini 2.5 is able to search the web, so if you're able to find the answer on reddit, maybe it can too.

Sure, but you can't cite this puzzle as proof that this model is "better than 95+% of the population at mathematical reasoning" when the method of solving (the "answer") it is online, and the model has surely seen it.

I think there’s a big push to train LLMs on maths problems - I used to get spammed on Reddit with ads for data tagging and annotation jobs.

Recently these have stopped and they’re now the ads are about becoming a maths tutor to AI.

Doesn’t seem like a role with long-term prospects.

It gets it wrong when you give it 728. It claims (728, 182, 546). I won't share the answer so it won't appear in the next training set.

with 728 the puzzle doesn't work since it's divisible by 8

But then the AI should tell you that, too, if it really understand the problem?

Fair, the question is what possible solutions exists.

> So it’s likely that it’s part of the training data by now.

I don't think this means what you think it means.

I did some interacting with the Tencent model that showed up here a couple days ago ( https://news.ycombinator.com/item?id=43447254 ):

> 你知道英国的 Steeleye Span 吗？

>> 是的，我知道英国传奇民谣摇滚乐队 Steeleye Span！他们是1970年代英国民谣复兴运动的重要代表之一，以融合传统英国民谣与摇滚乐器（如电吉他、鼓）的独特风格闻名。乐队核心成员包括 Maddy Prior（标志性女声）和 Tim Hart（多面手乐手），后加入的 Nigel Pegrum（传奇鼓手）也为他们的音效增添了厚重感。

>> 他们的经典作品包括：

>> - 《Gaudete》：一首中世纪拉丁文圣歌的震撼改编，曾在英国单曲榜冲到第14位；

>> - 《All Around My Hat》：同名专辑主打歌，成为民谣摇滚标杆；

>> - 《Cam Ye O'er Frae France》：对英伦历史的戏谑诠释。

>> 乐队的成员变动频繁，但始终保持着对传统音乐的创造性重构。近年来Maddy Prior仍活跃于音乐节，若想感受他们狂野的舞台张力，推荐从1970年代的黄金时期作品入手！需要具体推荐歌单或背景故事吗？ [happy blushing emoji]

> 我喜欢他们唱的 The Victory 那首歌。你会怎么描述？

>> 您提到的《The Victory》可能是指 Steeleye Span 的某首作品，但他们的正式专辑中并没有同名歌曲的记录。

>> [further suggestions omitted]

> 咦？Steeleye Span 的 Storm Force Ten 包括哪首歌？

>> 您提到的 《Storm Force Ten》 是 Steeleye Span 的第十张录音室专辑，发布于 1975年。这张专辑延续了乐队将传统英国民谣与硬摇滚融合的风格，同时融入了凯尔特、中世纪和叙事性元素，整体充满活力与戏剧张力。以下是专辑中的完整曲目列表及亮点解析：

>> 专辑曲目

>> 1.《Roll Right Stones》- 以巨石阵传说为灵感，开场的笛声与电吉他交织出神秘氛围，Maddy Prior 的嗓音如史诗吟诵。

>> 2.《Let Her Go Down》- 民谣基底搭配摇滚节奏，歌词暗喻航海冒险与命运抗争，鼓点密集如浪涌。

>> 3.《Hark! The Heron Voiced》- 中世纪风格叙事曲，描绘自然与灵性的对话，长笛与提琴的对话感极强。

>> 4.《The Brisk Young Widow》- 轻快的舞曲节奏，讲述寡妇与浪子间的诙谐故事，展现乐队幽默一面。

>> 5.《The Storm Force Ten》- 同名主打歌，以暴风雨比喻爱情冲突，吉他solo如闪电般撕裂，高潮部分极具爆发力。

>> 6.《The Green Knight》- 源自亚瑟王传说，骑士与自然的对抗主题，管风琴与吉他的音色营造出古老战场气息。

>> 7.《The Blacksmith》- 改编自传统民谣，重摇滚编曲让铁匠的锤击声化作节奏基石，Maddy 的即兴高音惊艳。

>> 8.《Cam Ye O’er Frae France》- 政治讽刺民谣，调侃英国王室历史，双吉他互飙如刀剑交锋。

>> 9.《The Maid of Llanwellyn》- 凯尔特风格抒情曲，弦乐编排如泣如诉，展现乐队柔情一面。

>> 10.《The Whisky Jacobites》- 庆祝苏格兰独立精神的狂欢曲，手风琴与摇滚乐器的碰撞充满节日气氛。

>> 专辑亮点

>> - 历史与传说的沉浸感：从亚瑟王到苏格兰起义，歌词如一部微型史诗。

>> - 音色实验性：管风琴、手风琴与电吉他混搭出独特的“民谣重金属”质感。

>> - 动态对比：快如风暴的摇滚段落与静谧的民谣叙事交替出现，张力十足。

>> 若您偏爱《The Victory》的恢弘感，推荐优先听 《The Storm Force Ten》 和 《The Green Knight》，这两首的史诗级编曲堪称乐队巅峰之作！需要试听链接或歌词解析吗？ [sunglasses emoji]

This is a question that obviously was in the training data. How do you get the answer back out of the training data?

Is there a reason for the downvotes here? We can see that having the answer in the training data doesn't help. If it's in there, what's that supposed to show?

It's entirely unclear what are you trying to get across, at least to me.

Generally speaking, posting output from a LLM, without explaining exactly what do you think it illustrates and why is frowned upon here. I don't think your comment does a great job of the latter.

>> So it’s likely that it’s part of the training data by now.

> I don't think this means what you think it means.

> I did some interacting with the Tencent model that showed up here a couple days ago [...]

> This is a question that obviously was in the training data. How do you get the answer back out of the training data?

What do I think the conversation illustrates? Probably that having the answer in the training data doesn't get it into the output.

How does the conversation illustrate that? It isn't subtle. You can see it without reading any of the Chinese. If you want to read the Chinese, Google Translate is more than good enough for this purpose; that's what I used.

> How does the conversation illustrate that? It isn't subtle. You can see it without reading any of the Chinese.

I can't, and I imagine most of the people who downvoted you couldn't either.

I think asking people to go to Google Translate to parse a random comment that seems to be 90% LLM output by volume is a bit much.

Your intentions are good, but your execution is poor.

I cannot figure out what the comment is trying to get across either. It's easy for you because you already know what you are trying to say. You know what the pasted output shows. The poor execution is in not spending enough time thinking about how someone coming in totally blind would interpret the comment.

No clue. Perhaps people object to the untranslated Chinese?

> Is there a reason for the downvotes here?

I didn't downvote you, but like (probably) most people here, I can't read Chinese; I can't derive whatever point you're trying to make just from with text you provided.

I have translated the Chinese. I still have no idea what point you're trying to make. You ask it questions about some kind of band, and it answers. Are you saying the answers are wrong?

And if it wasn’t, it is now

Thaks. I wanted to do exactly that: find the answer online. It is amazing that people (even in HN) think that LLM can reason. It just regurgitates the input.

I would say that 99.99% of humans do the same. Most people never come up with anything novel.

I would say maybe about 80% certainly not 99.99%. But I've seen that in college, some would only be able to solve the problems which were pretty much the same as others already seen. Notably some guys could easily come up with solutions to complex problems they did not see before. I have the opinion that no human at age 20 can have the amount of input a LLM today. And still humans of age 20 do come with very new ideas pretty often (new in the sense that (s)he has not seen that or anything like it before). Of course there are more and less creative/intelligent people...

Reasoning != coming up with something novel.

I think it can reason. At least if it can work in a loop ("thinking"). It's just that this reasoning is far inferior to human reasoning, despite what some people hastily claim.

Have you given a reasoning model a novel problem and watched its chain of thought process?

This whole answer hinges on knowing that 0 is not a positive integer, that's why I couldn't figure it out...