Evaluating Multimodal LLMs Using the Google IO 2024 Puzzle

Posted by simonbutt on Friday March 15, 2024 @03:10PM
from the hiresynth.ai dept.

https://hiresynth.ai/blog/googleio_puzzle_multimodal_eval.html

( 2 comments | 3 points | on HN )

Re:Evaluating Multimodal LLMs Using the Google IO 2024 Puzzle by malet on Friday March 15, 2024 @04:02PM (#39720078)

Surprising to see these models stumbling on what at first glance seems like a simple task, it would be interesting to see how the non-vision models fare if you convert the problems to ascii art

[ Parent | on HN ]

Re:Evaluating Multimodal LLMs Using the Google IO 2024 Puzzle by simonbutt on Friday March 15, 2024 @03:10PM (#39719557)

GPT-4V, Claude 3 Opus and Gemini Ultra go head to head in solving GoogleIO Puzzle 2024

[ Parent | on HN ]