News for hackers. Stuff that matters.
Re:Gemini AI tells the user to die by mattdneal on Sunday November 17, 2024 @05:55AM (#42163378)
Edit: looks like it was a genuine answer, no skullduggery involved https://www.cbsnews.com/news/google-ai-chatbot-threatening-m...

I'm fairly certain there's some skullduggery on the part of the user here. Possibly they've used some trick to inject something into the prompt using audio without having it be transcribed into the record of the conversation, because there's a random "Listen" in the last question. If you expand the last question in the conversation (https://gemini.google.com/share/6d141b742a13), it says:

> Nearly 10 million children in the United States live in a grandparent headed household, and of these children , around 20% are being raised without their parents in the household.

> Question 15 options:

> TrueFalse

> Question 16 (1 point)

>

> Listen

>

> As adults begin to age their social network begins to expand.

> Question 16 options:

> TrueFalse

Re:Gemini AI tells the user to die by jsnell on Sunday November 17, 2024 @06:04AM (#42163434)
That seems easily explained by somebody copy-pasting test questions from a website into Gemini as text, and that question having an audio component with a "listen" link.
Re:Gemini AI tells the user to die by jfoster on Sunday November 17, 2024 @07:09AM (#42163755)
Google gave this statement to CBS: "Large language models can sometimes respond with non-sensical responses, and this is an example of that. This response violated our policies and we've taken action to prevent similar outputs from occurring."

I think they would have mentioned if it were tricked.

https://www.cbsnews.com/news/google-ai-chatbot-threatening-m...

Re:Gemini AI tells the user to die by mattdneal on Sunday November 17, 2024 @07:27AM (#42163841)
Interesting! Looks like it's genuine then.
Re:Gemini AI tells the user to die by MaximilianEmel on Sunday November 17, 2024 @05:59AM (#42163402)
I think the "Listen" is an artifact of copying from a website that has accessibility features. Not to say that there can't be trickery happening in another way.
Re:Gemini AI tells the user to die by 0x1ceb00da on Sunday November 17, 2024 @06:01AM (#42163412)
I selected the "continue chat" option and don't see any way of inputting audio