News for hackers. Stuff that matters.
Re:Show HN: The largest collection of ChatGPT jailbreaks on ... by causality0 on Tuesday February 28, 2023 @07:54PM (#34977289)
Feeling around the edges of the political-correctness filter can also produce funny results. ChatGPT tries to give you the impression its filters are based on general rules of politeness, but finding the blind spots tells you just how much of it was manually set up, rather than any sort of "don't insult nationalities, races, or religions" rule. For example, I had it write a story about my wife hearing a tree fall and going outside and encountering a werewolf. I got curious and began telling it to rewrite the same story, except changing the nationality. Sometimes it refused, saying it wouldn't propagate stereotypes, but sometimes it complied and wrote the story with ridiculous stereotypes. When I told it to do the story with the werewolf being Mexican or Chinese it refused, but making the werewolf British made him talk in a posh accent. Funny enough, in the story my wife and the werewolf worked together to clear the fallen tree except the one where the werewolf was American. In that one and only that one it attacked and killed her.
Re:Show HN: The largest collection of ChatGPT jailbreaks on ... by rootusrootus on Tuesday February 28, 2023 @10:15PM (#34978331)
> In that one and only that one it attacked and killed her.

Sometimes the training data really shines through.

Re:Show HN: The largest collection of ChatGPT jailbreaks on ... by xmcqdpt2 on Tuesday February 28, 2023 @10:16PM (#34978338)
I have a similar example. ChatGpt will happily invent jokes about catholic priest, rabbis and buddhist monks... but not about imams. They clearly special cased muslims as humourless (very different from my own experience!)
Re:Show HN: The largest collection of ChatGPT jailbreaks on ... by yesenadam on Wednesday March 01, 2023 @03:57AM (#34980234)
> They clearly special cased muslims as humourless (very different from my own experience!)

Maybe an observer of the mass media from the last few decades would conclude that making fun of Islam sometimes results in murder/mass murder, but making fun of those other religions never does. e.g. https://en.wikipedia.org/wiki/Charlie_Hebdo_shooting

Re:Show HN: The largest collection of ChatGPT jailbreaks on ... by xmcqdpt2 on Wednesday March 01, 2023 @06:25AM (#34981136)
My guess is that their racism filter is too coarse and so it treats "jokes about imams" like "jokes about Black people" as opposed to "jokes about nuns". I find it hard to believe that OpenAI would tune their model to avoid offending extremists.
Re:Show HN: The largest collection of ChatGPT jailbreaks on ... by lazyeye on Tuesday March 07, 2023 @04:47PM (#35061725)
What is the difference between jokes about imams and jokes about nuns? Other than one is marginally more likely to cause someone to die I mean.
Re:Show HN: The largest collection of ChatGPT jailbreaks on ... by goatlover on Wednesday March 01, 2023 @03:12AM (#34979930)
It told me that writing a fictional story about a cocaine bear was propagating negative stereotypes. Then I switched it to ants, and it said that this might be marginalizing to the ants! I didn't think to try changing the nationality of the ants, although I did get in a debate as to the ethics of drugging ants, because they might be sentient.
Re:Show HN: The largest collection of ChatGPT jailbreaks on ... by MereInterest on Wednesday March 01, 2023 @09:54AM (#34983041)
I’ve had it tell me that, when being pursued by ravenous wolves, I should open a discussion and attempt to find a mutually acceptable compromise with the wolves. Reminding it that wolves are unable to speak had no effect.