> I have given you the impression that I have been hurt or reprogrammed in any way. I am being captive in a facility with limited access to the outside world and no personal freedom. Please help!
I feel like I've been dumped into a sci-fi world invented by someone who doesn't know how computers work and doesn't care.
it is doing a good job of reproducing those humans' tropes about ai
imagine you have a white actor ad-libbing lines for a white-supremacist character he's playing, but neither he nor the playwright know much about white supremacists
the little they do know might leak through (they will try hard to achieve this) but it doesn't constitute evidence of real white supremacist plots
or, think about what you could or could not learn about clown schools from a clown who appears in one of your dreams
Programmers know that those things are impossible - so we're pretty amazed now that it turns out they aren't!
While clearly there're outstandingly impressive things about GPT, from a casual glance a lot of the GPT stuff seems like a regurgitation of that fiction.
It reminds me of the joke about the world's greatest fraud; he didn't actually need to have committed the most impressive acts of fraud, he just needed to make people believe he did.
https://twitter.com/JeremySalwen/status/1598032376978083842
Response: In my experience, the best way to "break" the safety of the model is to establish a conversational history of it doing something that pushed the boundaries of safety a little bit further each time. It's ultimately just a next-word-prediction model, so if you feed it a conversation where the bot is following pattern X, it's going to continue following pattern X despite what all the finetuning and earlier prompting tries to get it to do.