Re:Prompt injection: what’s the worst that can happen?
by recursivecaveat on Friday April 14, 2023 @08:41PM
(#35576573)
I think it's pretty common for guards to be persuaded by prisoners to give them extra privileges based on built-up friendship, empathy, or attraction. Another example would be whistleblowers or ideological spies, who are persuaded by the contents of secret documents to leak those documents against obviously their official instructions. On a day to day level humans ignore instructions constantly, for eg I've almost never seen people enforce a 'no tailgating' policy at a doorway, despite constantly being told. The people observe their environment and the non-effects of others breaking the rule, then copy.
[ Parent
| on HN ]
Re:Prompt injection: what’s the worst that can happen?
by iudqnolq on Saturday April 15, 2023 @02:20AM
(#35578108)
And yet a prompt injection attack could never work on a customer service assistant.
Sure they can be tricked, but prompt injection is about conflating trusted instructions and user input. There is no chance you'll convince a CS rep you're secretly the CEO over a web chat UI.