Vision Language Models Are Biased

Re:Vision Language Models Are Biased by runako on Tuesday June 03, 2025 @02:05PM (#44172836)

FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this.

For example: "The animal in the image is a chicken, and it appears to have four legs. However, chickens normally have only two legs. The presence of four legs suggests that the image may have been digitally altered or artificially generated."

I don't have a good explanation for why I got different results.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by roywiggins on Tuesday June 03, 2025 @02:27PM (#44173078)

I gave ChatGPT some miswritten Braille a while ago and it completely, but confidently, messed it up. The sign reads "no smoking" but the braille doesn't. ChatGPT 1) read the English lettering first and then hallucinated the braille and the 2) when given only the braille, failed almost as hard. It even generated fake transcriptions in Unicode braille characters.

https://chatgpt.com/share/683f3e7d-0dfc-8005-b6c9-99e3d39ff4...

https://chatgpt.com/share/683f3e49-9c58-8005-99a6-c3a919838b...

[ Parent | on HN ]

Re:Vision Language Models Are Biased by Workaccount2 on Tuesday June 03, 2025 @02:43PM (#44173229)

This is hard to understand without the original images, it looks like OpenAI doesn't serve them in the share link.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by roywiggins on Tuesday June 03, 2025 @02:55PM (#44173354)

Annoying. The actual braille on the sign was "⠁⠒⠑⠎⠎⠊⠼" which I gather means "accessible" in abbreviated braille. None of my attempts got it to even transcribe it to Unicode characters properly. I got "elevator", "friend", etc. Just wildly making stuff up and completely useless, even when it wasn't distracted by the No Smoking sign (in the second case I cropped out the rest of the sign). And in all cases, supremely confident.

This seems like something a VLM should handle very easily, but instead I got pure nonsense.

https://www.facebook.com/share/p/12Gw55Gr2SZ/

[ Parent | on HN ]

Re:Vision Language Models Are Biased by dragonwriter on Tuesday June 03, 2025 @06:35PM (#44175514)

> This seems like something a VLM should handle very easily

Not if its training data doesn't include braille as first class but has lots of braille signage with bad description (e.g., because people assumed the accompanying English matches the braille.)

This could very well be the kind of mundane AI bias problem that the x-risk and tell-me-how-to-make-WMD concerns have shifted concerns about problems in AI away from.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by roywiggins on Tuesday June 03, 2025 @10:14PM (#44176630)

I'd wager that correctly labeled braille far exceeds dumb braille, and when presented with just the braille it flat out hallucinated braille characters that weren't there. It didn't seem to actually be parsing the dots at all. My theory is that it has hardly seen any braille, despite it insisting that it knows how to read it.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by inerte on Tuesday June 03, 2025 @02:57PM (#44173383)

I took a screenshot of the chicken, so low res, and got {4} https://chatgpt.com/share/683f4506-ae18-800f-8c27-5c5e91429a...

Also I think the authors used the API, and maybe there are differences between the API and chatgpt.com behavior...

[ Parent | on HN ]

Re:Vision Language Models Are Biased by simonw on Tuesday June 03, 2025 @04:05PM (#44174122)

ChatGPT is running a special model but it's also available through the API: https://platform.openai.com/docs/models/chatgpt-4o-latest

The system prompt may still make a difference though.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by runako on Tuesday June 03, 2025 @03:38PM (#44173799)

I could rant for quite a while about how OpenAI and Anthropic manage their apps vs their APIs. It's really quite strange that they both landed on the solution of non-public APIs that perform differently than their public APIs.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by anguyen8 on Tuesday June 03, 2025 @05:02PM (#44174702)

https://imgur.com/cO7eFNt

o3 Chat is also similarly wrong, saying {4}.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by vokhanhan25 on Tuesday June 03, 2025 @04:49PM (#44174571)

You should try with other models besides GPT-4o, because in the paper they also show that GPT4.1 (~GPT-4o) gives 4 legs instead of 2 legs.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by runako on Tuesday June 03, 2025 @10:48PM (#44176779)

I mean perhaps! But that would undermine the conclusion of the article.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by dwringer on Tuesday June 03, 2025 @06:14PM (#44175369)

Speculating, I would imagine that different prompts submitted along with the image might elicit wildly different behavior in how a multi modal VLM may respond to a given image, potentially affecting the relative tendency to upweight its effective inferences from prior training versus focusing more primarily on the new image itself.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by michaelt on Tuesday June 03, 2025 @06:52PM (#44175630)

> FWIW I tried the first couple of examples in ChatGPT 4o and couldn't replicate this.

I can replicate the flag examples from Figure 15 in the paper, if not the Adidas one from Figure 9: https://chatgpt.com/share/683f7c3a-b318-8011-9759-c495db2556... it even confirms its wrong answer when asked to check again.

[ Parent | on HN ]

Re:Vision Language Models Are Biased by obscurette on Wednesday June 04, 2025 @10:19AM (#44181005)

I suspect that responses are altered/corrected based on what people query from popular online models. I have had several occasions that I ask some "How do I ... in X software?" question some day and model keeps hallucinating nonexistant config options regardless how many times I keep saying "This option doesn't exist in software X". But if I asked the same question some days later, the answer was completely different and made even some sense.

[ Parent | on HN ]