News for hackers. Stuff that matters.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by doctor_eval on Saturday March 04, 2023 @05:27PM (#35025111)
This was the thing that totally blew me away when I was introduced to GPT a couple of months ago.

It’s “programmed” in natural language.

And when you think about it, it’s obvious: we don’t really know how it works, so how else could we program it?

That said, I don’t understand why the outputs of these systems aren’t “read” by a second GPT instance that is tasked with determining if the output is OK.

Re:Prompt to bypass the restrictions of Bing Chat or to rest... by Animats on Saturday March 04, 2023 @05:37PM (#35025195)
It does that, right? You get a response, generated word by word, and then the censorship classifier reads it and makes the entire response disappear.
Re:Prompt to bypass the restrictions of Bing Chat or to rest... by doctor_eval on Sunday March 05, 2023 @01:59AM (#35027506)
Yeah - it seems to, sometimes, but considering all these prompt leaks, it doesn’t seem terribly good. Like it seems obvious that Sydney could detect its own prompts, but it doesn’t.

I mean you could do that with grep. So what’s with that?