News for hackers. Stuff that matters.
Re:AI behavior guardrails should be public by stainablesteel on Wednesday February 21, 2024 @03:08PM (#39458824)
gemini seems to have problems generating white people and honestly this just opens the door for things that are even more racist [1], the harder you try the more you'll fail, just get over the DEI nonsense already

1. https://twitter.com/wagieeacc/status/1760371304425762940

Re:AI behavior guardrails should be public by AnarchismIsCool on Wednesday February 21, 2024 @04:43PM (#39459986)
I don't think the DEI stuff is nonsense, but SV is sensitive to this because most of their previous generation of models were horrifyingly racist if not teenage nazis, and so they turned the anti-racism knob up to 11 which made the models....racist but in a different way. Like depicting colonial settlers as native americans is extremely problematic in its own special way, but I also don't expect a statistical solver to grasp that context meaningfully.
Re:AI behavior guardrails should be public by grotorea on Wednesday February 21, 2024 @05:02PM (#39460266)
So you're saying in a way this is /pol/ and Tay's fault?
Re:AI behavior guardrails should be public by AnarchismIsCool on Wednesday February 21, 2024 @05:11PM (#39460364)
Looks around at everything ...is there anything that isn't 4chan's fault at this point?

Realistically, kinda. There have always been tons of anecdotes of video conference systems not following black people, cameras not white balancing correctly on darker faces etc. That era of SV was plagued by systems that were built by a bunch of young white guys who never tested them with anyone else. I'm not saying they were inherently racist or anything, just that the broader society really lambasted them for it and so they attempted to correct. Really, the pendulum will continue to swing and we'll see it eventually center up on something approaching sanity but the hyper-authoritarian sentiment that SV seems to have (we're geniuses and the public is stupid, we need to correct them) is...a troubling direction.

Re:AI behavior guardrails should be public by callalex on Wednesday February 21, 2024 @08:33PM (#39462017)
I’m always hesitant to jump straight to ringing the racism bell when it comes to problems that arise from fundamental physics. I have dark skin, and I constantly struggle with automatic faucets, soap dispensers, and towel dispensers in public bathrooms. None of those are made by big tech.
Re:AI behavior guardrails should be public by Jason_Protell on Wednesday February 21, 2024 @03:14PM (#39458887)
Is there any evidence that this is a consequence of DEI rather than a deeper technical issue?
Re:AI behavior guardrails should be public by flumpcakes on Wednesday February 21, 2024 @05:02PM (#39460265)
I don't understand how people could even argue that this is in any way acceptable. Fighting "bias" has become some boogyman and anything "non-white" is now beyond reproach. Shocking.
Re:AI behavior guardrails should be public by AnarchismIsCool on Wednesday February 21, 2024 @06:25PM (#39461106)
Fighting bias is a good thing, you'd have to be pretty...er...biased to believe otherwise. Bias is fundamentally a distortion or deviation from objective reality.

This, on the other hand, is just fucking stupid political showboating that's hurting their SV white knight cause. It's just differently flavored bias

Re:AI behavior guardrails should be public by gedy on Wednesday February 21, 2024 @05:41PM (#39460619)
Seriously, I've basically written off using Gemini for good after this HR style nonsense. It's a shame that Google, who invented much of this tech, is so crippled by their own people's politics.
Re:AI behavior guardrails should be public by gs17 on Wednesday February 21, 2024 @05:45PM (#39460656)
"I can't generate white British royalty because they exist, but I can make up black ones" is pretty close to an actually valid reason.
Re:AI behavior guardrails should be public by Jensson on Wednesday February 21, 2024 @03:22PM (#39458977)
You get 4 images per time and are lucky to get one white person when asked for it, no other model has that issue. Other models has no problems generating black people either, so it isn't that other models only generates white people.

So either it isn't a technical issue or Google failed to solve a problem everyone else easily solved. The chances of this having nothing to do with DEI is basically 0.

Re:AI behavior guardrails should be public by ceejayoz on Wednesday February 21, 2024 @03:40PM (#39459215)
Depending on how broadly you define it, something like 10-30% of the world's population is white. Africa is about 20% of the world population; Asia is 60% of it.

One in four sounds about right?

Re:AI behavior guardrails should be public by cm2012 on Wednesday February 21, 2024 @03:43PM (#39459259)
It does the same if you ask for pictures of past popes, 1945 German soldiers, etc.
Re:AI behavior guardrails should be public by ceejayoz on Wednesday February 21, 2024 @03:45PM (#39459293)
It'll also add extra fingers to human hands. Presumably that's not because of DEI guardrails about polydactyly, right?

The current state of the art in AI gets things wrong regularly.

Re:AI behavior guardrails should be public by cm2012 on Wednesday February 21, 2024 @03:48PM (#39459334)
Sure, but this one is from Google adding a tag to make every image of people diverse, not AI randomness.
Re:AI behavior guardrails should be public by ceejayoz on Wednesday February 21, 2024 @03:50PM (#39459358)
Am I missing something in the link demonstrating that, or is it conjecture?
Re:AI behavior guardrails should be public by cm2012 on Wednesday February 21, 2024 @04:26PM (#39459782)
It's been demonstrated on Twitter a few times, can't find a link handy
Re:AI behavior guardrails should be public by perlclutcher on Wednesday February 21, 2024 @04:30PM (#39459823)
https://twitter.com/altryne/status/1760358916624719938

Here's some corporate-lawyer-speak straight from Google:

> We are aware that Gemini is offering inaccuracies...

> As part of our AI principles, we design our image generation capabilities to reflect our global user base, and we take representation and bias seriously.

Re:AI behavior guardrails should be public by ceejayoz on Wednesday February 21, 2024 @04:38PM (#39459930)
That doesn't back up the assertion; it's easily read as "we make sure our training sets reflect the 85% of the world that doesn't live in Europe and North America". Again, 1/4 white people is statistically what you'd expect.
Re:AI behavior guardrails should be public by withinboredom on Wednesday February 21, 2024 @07:06PM (#39461398)
Fuck, this is going to sound fucked up... but just because you have a 1/4 chance of getting a random white person from the globe, they generally tend to clump together. For example, you generally find a shitload of Asian people in Asia, white people in Europe, and African people in Africa, and Indian people in India.

Probably the only chance where you wouldn't expect this are in heavily colonized places like South Africa, Australia, and the Americas.

Re:AI behavior guardrails should be public by u32480932048 on Wednesday February 21, 2024 @06:35PM (#39461191)
Sure, but I see three 200 responses and a 400 - not 1/4 white people as statistically expected.
Re:AI behavior guardrails should be public by int_19h on Wednesday February 21, 2024 @05:39PM (#39460598)
If you look closely at the response text that accompanies many of these images, you'll find recurring wording like "Here's a diverse image of ... showcasing a variety of ethnicities and genders". The fact that it uses the same wording strongly implies that this is coming out of the prompt used for generation. My bet is that they have a simple classifier for prompts trained to detect whether it requests depiction of a human, and appends "diverse image showcasing a variety of ethnicities and genders" to the prompt the user provided if so. This would totally explain all the images seen so far, as well as the fact that other models don't have this kind of bias.
Re:AI behavior guardrails should be public by xdennis on Wednesday February 21, 2024 @06:47PM (#39461271)
OpenAI has no problem showing accurate pictures. You know it's Google-induced bias, but feign ignorance.

If you ask for a picture of nazi soldiers it shouldn't have 60% Asian people like you say. You know you're wrong but instead of admitting it, you're moving the goalpost to "hands".

This entire thread is you being insincere.

Re:AI behavior guardrails should be public by strangeattractr on Wednesday February 21, 2024 @07:11PM (#39461430)
Have you bothered to look at all? Read the output of the model when asked about why it has the behaviour it does. Look at the plethora of images it generates that are not just historically inaccurate but absurdly so. It tells you "heres a diverse X" when you ask for X. Yet asking for pictures of Koreans generates only Asian people but prompts for Scots or French people in historical periods generate mostly non-white people. You're being purposefully obtuse, Google has had racism complaints about previous models, talks often about AI safety and avoiding 'bias'. You're trying to argue that it's more likely that the training data had an inherent bias against generating white people in images purely by chance?
Re:AI behavior guardrails should be public by thepasswordis on Wednesday February 21, 2024 @05:01PM (#39460239)
The AI will literally scold you for asking it to make white characters, and insists that you need to be inclusive and that it is being intentionally dishonest to force the issue.
Re:AI behavior guardrails should be public by lmm on Wednesday February 21, 2024 @05:13PM (#39460379)
This specific thing is a much more blatant class of error, and one that has been known to occur in several previous models because of DEI systems (e.g. in cases where prompts have been leaked), and has never been known to occur for any other reason. Yes, it's conceivable that Google's newer, beter-than-ever-before AI system somehow has a fundamental technical problem that coincidentally just happens to cause the same kind of bad output as previous hamfisted DEI systems, but come on, you don't really believe that. (Or if you do, how much do you want to bet? I would absolutely stake a significant proportion of my net worth - say, $20k - on this)
Re:AI behavior guardrails should be public by ceejayoz on Wednesday February 21, 2024 @05:31PM (#39460538)
> has never been known to occur for any other reason

Of course it has. Again, these things regularly give humans extra fingers and arms. They don't even know what humans fundamentally look like.

On the flip side, humans are shitty at recognizing bias. This comment thread stems from someone complaining the AI only rarely generated white people, but that's statistically accurate. It feels biased to someone in a majority-white nation with majority-white friends and coworkers, but it fundamentally isn't.

I don't doubt that there are some attempts to get LLMs to go outside the "white westerner" bubble in training sets and prompts. I suspect the extent of it is also deeply exaggerated by those who like to throw around woke-this and woke-that as derogatories.

Re:AI behavior guardrails should be public by lmm on Wednesday February 21, 2024 @05:58PM (#39460816)
> Of course it has. Again, these things regularly give humans extra fingers and arms. They don't even know what humans fundamentally look like.

> This comment thread stems from someone complaining the AI only rarely generated white people, but that's statistically accurate. It feels biased to someone in a majority-white nation with majority-white friends and coworkers, but it fundamentally isn't.

So the AI is simultaneously too dumb to figure out what humans look like, but also so super smart that it uses precisely accurate racial proportions when generating people (not because it's been specifically adjusted to, but naturally)? Bullshit.

> I don't doubt that there are some attempts to get LLMs to go outside the "white westerner" bubble in training sets and prompts. I suspect the extent of it is also deeply exaggerated by those who like to throw around woke-this and woke-that as derogatories.

You're dodging the question. Do you actually believe the reason that the last example in the article looks very much not like a man is a deep technical issue, or a DEI initiative? If the former, how much are you willing to bet? If the latter, why are you throwing out these insincere arguments?

Re:AI behavior guardrails should be public by samatman on Wednesday February 21, 2024 @07:04PM (#39461384)
A very impressive display of crimestop you've got going in this thread. How did you end up like this?
Re:AI behavior guardrails should be public by pb7 on Wednesday February 21, 2024 @08:00PM (#39461780)
Congraulations, here is your gold medal in mental gymnastics. Enough now.

It literally refuses to generate images of white people when prompted directly while not only happily obliging but only producing that specific race in all 4 results. It’s discriminatory and based on your inability to see that, you may be too.

Re:AI behavior guardrails should be public by wruza on Wednesday February 21, 2024 @06:28PM (#39461131)
If it does, shouldn't there be 60% asians?
Re:AI behavior guardrails should be public by minimaxir on Wednesday February 21, 2024 @03:53PM (#39459401)
When DALL-E 2 was released in 2022, OpenAI published an article noting that the inclusion of guardrails was a correction for bias: https://openai.com/blog/reducing-bias-and-improving-safety-i...

It was widely criticized back then: the fact that Google both brought it back and made it more prominent is weird. Notably, OpenAI's implementation is more scoped.

Re:AI behavior guardrails should be public by allmadhare22 on Wednesday February 21, 2024 @04:29PM (#39459813)
Depending on what you ask for, it injects the word 'diverse' into the response description, so it's pretty obvious they're brute forcing diversity into it. E.g. "Generate me an image of a family" and you will get back "Here are some images of a diverse family".
Re:AI behavior guardrails should be public by nickthegreek on Wednesday February 21, 2024 @03:32PM (#39459113)
I dont think so. My boss wanted me to generate a birthday image for a co-worker of a John Cena flyfishing. ChatGPT refused to do so. So I had to move to describing the type of person John Cena is instead of using his name. I kept giving me bearded people no matter what. I thought this would be the perfect time to try out Gemini for the first time. Well shit, It wont even give me a white guy. But all the black dudes are beardless.

update: google agrees there is an issue. https://news.ycombinator.com/item?id=39459270

Re:AI behavior guardrails should be public by 8f2ab37a-ed6c on Wednesday February 21, 2024 @03:57PM (#39459442)
It feels that the image generation it offers is perfect for some sort of a California-Corporate Style, e.g. you ask it for a "photo of people at the board room" or "people at the company cafeteria" and you get the corporate friendly ratio of colors, ability-levels, sizes etc. See Google's various image assets: https://www.google.com/about/careers/applications/ . It's great for coastal and urban marketing brochures.

But then then same California Corporate style makes no sense for historical images, so perhaps this is where Midjourney comes in.

Re:AI behavior guardrails should be public by mike_d on Wednesday February 21, 2024 @04:05PM (#39459545)
It is possible Google tried to avoid likenesses of well known people by removing any image from the training data that contained a face and then including a controlled set of people images.

If you give a contractor a project that you want 200k images of people who are not famous, they will send teams to regions where you may only have to pay each person a few dollars to be photographed. Likely SE Asia and Africa.

Re:AI behavior guardrails should be public by 123yawaworht456 on Wednesday February 21, 2024 @03:30PM (#39459082)
yes, there's irrefutable evidence that models are wrangled into abiding the commissars' vision rather than just do their job and output the product of their training data.

https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf

Re:AI behavior guardrails should be public by xdennis on Wednesday February 21, 2024 @06:35PM (#39461196)
It's not just Gemini, it's Google. And old example is to just search "white people" on Google Images. Almost all the results are black people. https://www.google.com/search?q=white+people&tbm=isch&hl=ro