foddiangames
Re:VideoGameBench from Princeton: Can vision-language models play 90s ...
on Thursday May 29, 2025 @02:21PM (#44128740)
Wow so without scaffolding the LLMs can't solve any of these games... Super cool work!
Re:Claude, GPT, Gemini Play Doom II and 19 More (VideoGameBench)
on Friday April 18, 2025 @10:53AM (#43728643)
Researchers from Princeton introduce a research preview of VideoGameBench, a benchmark which challenges vision-language models to complete, in real-time, a suite of 20 different popular video games from both hand-held consoles and PC.
GPT-4o, Claude Sonnet 3.7, Gemini 2.5 Pro, and Gemini 2.0 Flash playing Doom II (default difficulty) on VideoGameBench-Lite with the same input prompt! Models achieve varying levels of success but none are able to pass even the first level.