News for hackers. Stuff that matters.
Re:Vision Language Models Are Biased by 0xab on Tuesday June 03, 2025 @09:00PM (#44176315)
> When VLMs make errors, they don't make random mistakes. Instead, 75.70% of all errors are "bias-aligned" - meaning they give the expected answer based on prior knowledge rather than what they actually see in the image.

Yeah, that's exactly what our paper said 5 years ago!

They didn't even cite us :(

"Measuring Social Biases in Grounded Vision and Language Embeddings" https://arxiv.org/pdf/2002.08911

Re:Vision Language Models Are Biased by 3abiton on Tuesday June 03, 2025 @11:14PM (#44176891)
It's easier to succeed if you ignore the issues, andthe users are not aware of it.the rate of evolution of "AI" recently is so fast, no one is stopping to do actual benchmarks and analysis of allyhe new models.
Re:Vision Language Models Are Biased by moralestapia on Tuesday June 03, 2025 @10:31PM (#44176701)
That's weird, you're at MIT. You're in the circle of people that's allowed to succeed.

I wouldn't think much about it, as it was probably a genuine mistake.

Re:Vision Language Models Are Biased by JackYoustra on Tuesday June 03, 2025 @10:52PM (#44176792)
What does allowed to succeed mean?
Re:Vision Language Models Are Biased by moralestapia on Tuesday June 03, 2025 @10:58PM (#44176828)
Your work usually has 1,000x the exposure and external validation compared to doing it outside those environments, where it would just get discarded and ignored.

Not a complain, though. It's a requirement for our world to be the way it is.

Re:Vision Language Models Are Biased by _345 on Tuesday June 03, 2025 @11:34PM (#44176990)
Is there truth to this? Do you have any sources to link to on this
Re:Vision Language Models Are Biased by moralestapia on Wednesday June 04, 2025 @02:22PM (#44183856)
Sure dude, here's the link to the UN Resolution about which researchers deserve attention and which others do not, signed by all countries around the world [1].

*sigh*

It's pretty obvious, if you publish something at Harvard, MIT, et. al. you even get a dedicated PR team to make your research stand out.

If you publish that on your own, or on some small research university in Namibia, no one will notice.

I might be lying, though, 'cause there's no "proof".

1: https://tinyurl.com/3uf7r5r7

Re:Vision Language Models Are Biased by hkmaxpro on Wednesday June 04, 2025 @12:37AM (#44177276)
I think social biases (e.g. angry black women stereotype) in your paper is different from cognitive biases about facts (e.g. number of legs, whether lines are parallel) that OP is about.

Social biases are subjective. Facts are not.

Re:Vision Language Models Are Biased by rcxdude on Wednesday June 04, 2025 @05:13AM (#44178695)
As far as the model's concerned, there's not much difference. Social biases will tend to show up objectively in the training data because the training data is influenced by those biases (the same thing happens with humans, which how these biases can proliferate and persist).
Re:Vision Language Models Are Biased by vokhanhan25 on Wednesday June 04, 2025 @06:34AM (#44179237)
I see a clear difference. One is objective (only one correct answer), one is subjective (multiple plausible answers)
Re:Vision Language Models Are Biased by ramblerman on Wednesday June 04, 2025 @03:34AM (#44178124)
What do you genuinely think they built upon from your paper?

If anything, the presentation of their results in such an accessible format next to the paper should be commended.

Re:Vision Language Models Are Biased by EvgeniyZh on Wednesday June 04, 2025 @04:21AM (#44178428)
Well you send a vaguely worded email like "I think you may find our work relevant" and everyone knows what that means and adds the citation
Re:Vision Language Models Are Biased by anguyen8 on Thursday June 05, 2025 @12:09PM (#44193013)
Hello 0xab,

Sorry that we missed your work. There are a lot of works in this area both textual and visual, especially social biases.

We wish to mention all but the space is limited so one can often discuss the most relevant ones. We'll consider discussing yours in our next revision.

Genuine question: Would you categorize the type of bias in our work "social"?