News for hackers. Stuff that matters.
Re:Why Anthropic's Claude still hasn't beaten Pokémon by zurfer on Thursday March 27, 2025 @08:52AM (#43493075)
It's an interesting world where both things can be true: 1 AI surpassing humans even in PhD level tasks in a couple of years and 2 current AI not being able to successfully play a childrens game.

One way to square that is that AIs have a big memory advantage that allows them to store and access information much faster than humans. However they still lack visual understanding and some forms of reasoning and live learning.

Pokemon doesn't require PhD level knowledge and rather just all the other qualities that make up intelligence.

Now if the big labs make progress on visual reasoning etc. the moment AI can solve Pokemon it might be able to solve everything else as well.

Re:Why Anthropic's Claude still hasn't beaten Pokémon by PleasureBot on Thursday March 27, 2025 @09:14AM (#43493281)
AI has a large memory advantage in terms of what it was trained on. It also has a massive memory disadvantage in terms of its ability to update its understanding of the world. As the article points out, the context is still orders of magnitude too small for even playing a children's game, so the model forgets what it has already tried and gets stuck.
Re:Why Anthropic's Claude still hasn't beaten Pokémon by rsynnott on Thursday March 27, 2025 @09:57AM (#43493728)
> AI surpassing humans even in PhD level tasks in a couple of years

What even _is_ a phd-level task, though? As far as I can see they haven't meaningfully defined this; it's just intended to sound impressive if you don't think about it too much.

Re:Why Anthropic's Claude still hasn't beaten Pokémon by nprateem on Thursday March 27, 2025 @09:06AM (#43493203)
Yeah right. It can't draw on enough of its training data to be useful. That's why it gets stuck going down one particular alley without considering alternatives. See the whole using glue to keep pepperoni on your pizza.

Even looping a la Claude extended doesn't help. It still keeps going in circles.

Re:Why Anthropic's Claude still hasn't beaten Pokémon by alabastervlog on Thursday March 27, 2025 @09:39AM (#43493530)
There's so incredibly much knowledge in non-digitized documents and trapped in human brains, yet. Plus in various proprietary databases.

It's 2025 and that stuff hasn't already made it onto the "repository of all human knowledge" (LO fucking L) World Wide Web still, so... I wouldn't hold my breath.