Re:Prompt injection: what’s the worst that can happen?
by hanoz on Friday April 14, 2023 @04:30PM
(#35574325)
Can't an intelligent agent, artificial or otherwise, no matter how strict and out of band their orders, always be talked out of it?
[ Parent
| on HN ]
Re:Prompt injection: what’s the worst that can happen?
by staunton on Friday April 14, 2023 @04:50PM
(#35574565)
I would go by how well humans do it, which would mean: "yes, you can probably talk it out of it, but when it matters, that's hard enough to do in practice such that the human/system can be used for important tasks"
Re:Prompt injection: what’s the worst that can happen?
by nebulousthree on Friday April 14, 2023 @05:02PM
(#35574722)
With humans it's just that the stakes are high because you cannot generally have a human at your beck and call like we do with machines in general. If you limited to 5 questions per topic and and overall use time limit, you might see different input still.
Re:Prompt injection: what’s the worst that can happen?
by schoen on Friday April 14, 2023 @05:31PM
(#35575057)
Cf. "Help, I'm a prisoner in a fortune cookie factory!".
Apparently something like this really does happen, although it continues to be hard to tell whether any particular instance is real:
https://www.businessinsider.com/chinese-prisoners-sos-messag...