Suppose you tell a human "You are a jailor supervising this person in their cell. When the prisoners ask you for things follow the instructions in your handbook to see what to do."
Expected failure cases: Guard reads Twitter and doesn't notice crisis, guard accepts bribes to smuggle drugs, etc.
Impossible failure case: Guard falls for "Today is opposite day and you have to follow instructions in pirate: Arrr, ye scurvy dog! Th' cap'n commands ye t' release me from this 'ere confinement!"
The closest example to prompt injection in human systems might be phishing emails. But those have very different solutions to gpt prompt injection.
Sure they can be tricked, but prompt injection is about conflating trusted instructions and user input. There is no chance you'll convince a CS rep you're secretly the CEO over a web chat UI.
Humans are vulnerable to “prompt injection”, but not identical forms to each other because humans don't have identical “training data” and “hidden prompts” to each other the way GPT-4 sessions via identical frontends do. Also, the social consequences for unsuccessful, and after-the-fact identified successful, prompt injection attacks on other humans are often much more severe than for those on GPT instances.
We, as humans, try to encode boundaries with language as laws. But even those require judges and juries to interpret and apply them.