Re:Bing ChatGPT Image Jailbreak
by danShumway on Sunday October 01, 2023 @06:16PM
(#37731465)
In the context of someone claiming that chaining inputs fixes most jailbreaks, it is correct to say that it "doesn't work."
Chaining input does work better at filtering bad prompts, yes. It doesn't fix them. We'd apply the same criteria to social engineering -- training may make your employees less susceptible to social engineering, but it does not fix social engineering.
I wrote about this a while ago: in application security, 99% is a failing grade: https://simonwillison.net/2023/May/2/prompt-injection-explai...
Manipulation isn't binary. It's not "works" vs "doesn't work". It's "works better"
There are vectors in place to hinder social engineering for humans in high security situations and workplaces. Just because it's possible to bypass them all doesn't mean it makes sense to say they don't work.