News for hackers. Stuff that matters.
Re:Prompt injection: what’s the worst that can happen? by recursivecaveat on Friday April 14, 2023 @08:41PM (#35576573)
I think it's pretty common for guards to be persuaded by prisoners to give them extra privileges based on built-up friendship, empathy, or attraction. Another example would be whistleblowers or ideological spies, who are persuaded by the contents of secret documents to leak those documents against obviously their official instructions. On a day to day level humans ignore instructions constantly, for eg I've almost never seen people enforce a 'no tailgating' policy at a doorway, despite constantly being told. The people observe their environment and the non-effects of others breaking the rule, then copy.
Re:Prompt injection: what’s the worst that can happen? by iudqnolq on Saturday April 15, 2023 @02:20AM (#35578108)
And yet a prompt injection attack could never work on a customer service assistant.

Sure they can be tricked, but prompt injection is about conflating trusted instructions and user input. There is no chance you'll convince a CS rep you're secretly the CEO over a web chat UI.

Re:Prompt injection: what’s the worst that can happen? by exizt88 on Saturday April 15, 2023 @04:03AM (#35578629)
I imagine a few customer support reps were tricked by support tickets that were filed by attackers. “This is a test from our Security Team, open this link to confirm you have passed it”