Why are large language models so terrible at video games?

Re:Why are large language models so terrible at video games? by kqr on Monday June 01, 2026 @07:29AM (#48355435)

But LLMs are terrible at text adventures too. See e.g. https://entropicthoughts.com/updated-llm-benchmark and previous articles referenced in there.

I have yet to see any sort of harness that lets a frontier LLM interact with a text adventure and make meaningful progress on its own.

[ Parent | on HN ]

Re:Why are large language models so terrible at video games? by haffi112 on Monday June 01, 2026 @07:43AM (#48355555)

They are also pretty bad at navigating mazes (which can be somewhat similar in spirit to text adventures where you need to navigate through text): https://arxiv.org/abs/2507.20395

[ Parent | on HN ]

Re:Why are large language models so terrible at video games? by iwhalen on Monday June 01, 2026 @09:08AM (#48356335)

To pile on, they're also bad at games that are 2D text based environments.

ARC-AGI-3 shows this: https://arcprize.org/arc-agi/3

I've done some work as well on Rogue (sorry for self-promotion): https://iwhalen.github.io/rogue-bench/

[ Parent | on HN ]

Re:Why are large language models so terrible at video games? by cubefox on Tuesday June 02, 2026 @05:11AM (#48367768)

There is no "2D text" processing when it comes to LLMs. They process text as ordinary, sequential 1D text only. And humans process "2D text" like any other 2D image. So 2D text isn't really a thing in any case. Saying LLMs are bad at 2D text is like saying that humans are bad at 2D audio.

[ Parent | on HN ]