News for hackers. Stuff that matters.
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by glimshe on Friday February 23, 2024 @03:12AM (#39478127)
In a nutshell for those without a X account: the problem here isn't that Google's AI gave "woke" results. The big problem here is that even one of the biggest AI specialists in the world, Google, wasn't able to fully predict or control the output of its AI. If we ever create a truly powerful, superintelligent AI under the assumption that we'll be able to control it, this incident shows that even a minor mistake from the AI builders could lead to the AI freeing itself from our control.
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by spamuel on Friday February 23, 2024 @03:22AM (#39478195)
This seems like a reaching explanation to try to wave away an ideological embarrassment as not being a consequence of the ideology.

"No one could have predicted the outcome of the instruction set we put in - which was followed precisely and in predictable fashion."

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by DaedPsyker on Friday February 23, 2024 @11:31AM (#39482714)
I don't agree with that framing. It reads more as saying what Google did was worse than it appears, not waving away.

Whatever your own opinion, Google did it out of what they perceived to be good intentions (and very likely business sense given a global audience for their products). Yet their intentions directly lead to unintended consequences. Google is being a baby with a gun in essence. Like he says, what if they decide to ask it to solve climate change and it decides to wipe humans out?

Obviously it's still very theoretical and can't do anything like that, but the point is more that perhaps Google doesn't have the culture necessarily to truly interrogate their actions.

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by therealjumbo on Friday February 23, 2024 @11:51AM (#39483002)
Here's his central point:

>This event is significant because it is major demonstration of someone giving a LLM a set of instructions and the results being totally not at all what they predicted.

Replace LLM with computer in that sentence, is it still novel? Laughably far from it, unexpected results are one of the defining features of moderately complex software programs going all the way back to the first person to program a computer. Some of the results are unexpected, but a lot are not, because it's literally doing what the prompt injection tells it to. Which isn't all that surprising but sure anyway...

>Obviously it's still very theoretical and can't do anything like that, but the point is more that perhaps Google doesn't have the culture necessarily to truly interrogate their actions.

Oh that's definitely true.

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by rayiner on Saturday February 24, 2024 @09:25AM (#39491691)
> Whatever your own opinion, Google did it out of what they perceived to be good intentions (and very likely business sense given a global audience for their products)

That makes even less sense, because most countries “globally” are internally quite homogenous. If someone in Bangladesh or China writes “show me pictures of people walking outside,” it’s even more jarring to deliberately insert random Latinos, East Asians, and Africans.

Given Google’s global audience, it might want to detect the customer’s location and show Chinese people pictures of Chinese people, and Japanese people pictures of Japanese people. That actually makes a lot of sense. But that’s not what they did.

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by hollerith on Saturday February 24, 2024 @10:19AM (#39492105)
In other words, even though it tried to be inclusive, a US company ended up being US-centric in that random Latinos, East Asians, and Africans are what you are likely to see when walking around the US, but not most of the rest of the world :)
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by monsecchris on Friday February 23, 2024 @03:42AM (#39478335)
The woke results were by design. This was not a mistake or unpredictable.
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by 4bpp on Friday February 23, 2024 @04:12AM (#39478532)
Are you contending that the black Wehrmacht soldiers, or black British kings eating watermelon, were anticipated or desired by Google staff?

Even if you model them as the most extreme conceivable woke boogeymen, surely they wouldn't want a scenario where the diversity programming produced a result that they would consider more insulting to minorities than the one that would have been produced without it.

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by weweersdfsd on Friday February 23, 2024 @06:26AM (#39479325)
The Google staff responsible were clearly lacking imagination. I think next time Google should consider hiring a QA team consisting of teenagers from 4chan, I think they would excel at the job of figuring out ways to generate offensive content with the app.

Of course even better would be utilizing more balanced training data, so you don't need silly and badly working prompt engineering to force more diversity.

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by lostmsu on Friday February 23, 2024 @12:31PM (#39483571)
"By design" is not the same as "what we wanted".
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by 4bpp on Friday February 23, 2024 @02:24PM (#39485009)
That's precisely what is typically considered to be the problem of AI alignment.
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by 15457345234 on Friday February 23, 2024 @05:47PM (#39487126)
> want a scenario where the diversity programming produced a result that they would consider more insulting to minorities

They're trolls. They don't care about minorities. They just use diversity to wind people up because then they can press the 'white fragility' button when people complain and segue into another level of trolling. If minorities get offended they'll attack them with 'uncle tom' and 'oreo' type insults.

You are not dealing with people operating in good faith, it's obvious. Expecting their actions to make sense is pointless. They just love drama. Performance artists. Theater kids. Somehow they've got in there and have formed a little nucleus of toxicity which is why the company's headed downhill so fast.

Someone alludes to it here:

https://news.ycombinator.com/item?id=39485537

And I've seen this happen. You hire one, they get some leverage to hire their buddies, before you know it they've set up their own little clown show and nobody wants to go anywhere near them. Then the wrecking starts.

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by thinkingemote on Friday February 23, 2024 @04:35AM (#39478672)
In amazed that the very large corporation didn't have a competent red team doing simple prompt tomfoolery to discover this.

After all prompt shenanigans are a well known issue.

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by caekislove on Saturday February 24, 2024 @05:27PM (#39495702)
It's Google, so they probably consider just the term "red team" to be "problematic".
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by vidarh on Friday February 23, 2024 @05:21AM (#39478973)
I mean, in this case it seems to involve a staggering level of lack of QA.
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by codewench on Friday February 23, 2024 @07:41AM (#39479826)
To be fair, that is at least consistent with Google's past history
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by kgeist on Friday February 23, 2024 @05:24AM (#39478990)
>this incident shows that even a minor mistake from the AI builders could lead to the AI freeing itself from our control.

There's most likely a hidden prompt which tells Gemini to make people racially diverse. And Gemini complied perfectly. I can't call it "freeing itself from our control". It did exactly what was asked.

It's just a bug which was introduced by software engineers. No one says "Windows freed itself from our control" when BSOD happens. It's just a bug.

Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by gmerc on Friday February 23, 2024 @05:33AM (#39479028)
This. We know this from the dumped System Prompt for Dall-e
Re:Yishan Wong: "Google's Gemini issue is not about woke/DEI" by kgeist on Friday February 23, 2024 @05:41AM (#39479077)
Not to say, miscommunication/misunderstanding happens between people all the time. And software has already killed/injured people without any AI involved. Nothing specific to AI here. It's more about a lack of adequate QA.