Re:Delimiters won’t save you from prompt injection
by r13a on Saturday May 13, 2023 @12:39PM
(#35930040)
> Doesn't it shift the risk
> instead of eliminating it?
Yes it's exactly that.
Of course I'm not trying to argue that there's a magic stick to make prompt injection just go away. My point is that prompt injection is so dangerous because we're letting the user directly interact with such a powerful beast as SOTA LLM.
By filtering prompts and answers by much less powerful but more specialized models we are heavily mitigating risks. But injection risks will still be there just not as wide open injections avenues as today.
Yes it's exactly that.
Of course I'm not trying to argue that there's a magic stick to make prompt injection just go away. My point is that prompt injection is so dangerous because we're letting the user directly interact with such a powerful beast as SOTA LLM.
By filtering prompts and answers by much less powerful but more specialized models we are heavily mitigating risks. But injection risks will still be there just not as wide open injections avenues as today.