Re:Kimi K3, and what we can still learn from the pelican ben...
by program_whiz on Friday July 17, 2026 @01:55PM
(#48950227)
This is a sight-reading test. If a musician practices a piece for thousands of hours, it would no longer be an effective sight reading / creativity test. The purpose of the test was to see how models would compose something novel requiring the ability to compose orthogonal, normally unrelated, components into a coherent image.
Re:Kimi K3, and what we can still learn from the pelican ben...
by alexjplant on Friday July 17, 2026 @02:16PM
(#48950453)
We do. People who, for example, memorize question banks to pass certification tests without knowing the underlying material are equally frowned upon for not having the problem solving skills that they purport to. I'll leave the contrasts between LLMs and people to the well-written sibling comments.
Re:Kimi K3, and what we can still learn from the pelican ben...
by computably on Friday July 17, 2026 @01:21PM
(#48949835)
Except, of course, LLMs are not humans, and they do not learn or "reason" in a way which even remotely resembles humans.
Plus obviously humans can still overfit to a specific style of test.
"That artist saw a pelican at the beach once!" [cue the outrage] "He's not a real artist, he's a cheater and produces nothing original!"