News for hackers. Stuff that matters.

A comment on Show HN: r1_vlm – Open-Source Framework for Visual Reasoning with GRPO. Parent comment

Re:Show HN: r1_vlm – Open-Source Framework for Visual Reason... by xoofoog on Saturday March 08, 2025 @11:53AM (#43301552)
What do you mean LLMs are bad at images? GPT or Claude can read text perfectly, and describe what's in a picture in a lot of detail. I feel like replacing OCR is one of the few things you can actually trust them for.
[ Parent | on HN ]
Re:Show HN: r1_vlm – Open-Source Framework for Visual Reason... by skumar17 on Saturday March 08, 2025 @11:58AM (#43301597)
That’s a good observation. For this project, I found that while the base model could “read” the image, it didn’t really understand how to use it. GRPO allowed it to effectively search the solution space.
[ Parent | on HN ]
Re:Show HN: r1_vlm – Open-Source Framework for Visual Reason... by oofbey on Saturday March 08, 2025 @12:15PM (#43301719)
That's true - they are quite good at OCR. But they're really bad at a bunch of tasks that seem like they should be super simple. Like "are these lines crossed" or "which letter is circled". See https://vlmsareblind.github.io/ for some clear examples.
[ Parent | on HN ]