News for hackers. Stuff that matters.
Re:Karpathy's Pelican by darrinm on Sunday August 02, 2026 @08:00PM (#49149666)
A simple prompt that still stumps frontier LLMs most of the time is “create a pinball game”. They’ll put all the right pieces there but then fail to arrange them such that the game is truly playable. They’ll put a wall in the way of the launch chute so the ball can’t be launched. Or the flippers will pivot the wrong way. Or there will be holes such that the ball drops off the bottom without getting within reach of the flippers, etc.

Opus 5 is the first I’ve seen to “one shot” it (in a harness, so it was more than one LLM call).

Re:Karpathy's Pelican by Schlagbohrer on Monday August 03, 2026 @02:56AM (#49152116)
Failed demos like this give weight to the argument that AIs need more of a world model, an understanding of how physics works to avoid obvious stumbles like this.
Re:Karpathy's Pelican by kzrdude on Monday August 03, 2026 @06:14AM (#49153704)
We don't use these models by way of "one shot" so I don't see why it's relevant. It's clearly useful to let them iterate.
Re:Karpathy's Pelican by yencabulator on Wednesday August 05, 2026 @01:00PM (#49185593)
No tool-calling LLM is really "one shot" anymore, so we can easily repurpose that expression to mean "without further human feedback". The generated pinball games still seem to end up coming out broken. There's not much conversation about a good arrangement of pinball playing fields, and the LLM is a next token predictor. Pick any out-of-mainstream topic and same happens. Pelicans on bicycles used to be out-of-mainstream.