Applications should not use user input for the system role. It's still not a firewall, but it's substantially better than the completion model from GPT3.
There was also a blog post / article / quite somewhere from OpenAI talking about how RL for GPT4 made it treat the system role as more immutable than was true in 3.5, but I'm not finding it in a quick search.
As the technology matures, we'll see security improvements as well. That's kind of the story of tech, right? SQL is doing pretty well despite having a similar problem with instructions versus user data.
I won't hang my hat on LLMs ever being perfect, but nor will I assert they are fundamentally broken and unfixable in this area. It is a very very young technology.
Applications should not use user input for the system role. It's still not a firewall, but it's substantially better than the completion model from GPT3.
There was also a blog post / article / quite somewhere from OpenAI talking about how RL for GPT4 made it treat the system role as more immutable than was true in 3.5, but I'm not finding it in a quick search.
As the technology matures, we'll see security improvements as well. That's kind of the story of tech, right? SQL is doing pretty well despite having a similar problem with instructions versus user data.
I won't hang my hat on LLMs ever being perfect, but nor will I assert they are fundamentally broken and unfixable in this area. It is a very very young technology.