News for hackers. Stuff that matters.
Re:Prompt injection: what’s the worst that can happen? by hanoz on Friday April 14, 2023 @04:30PM (#35574325)
Can't an intelligent agent, artificial or otherwise, no matter how strict and out of band their orders, always be talked out of it?
Re:Prompt injection: what’s the worst that can happen? by staunton on Friday April 14, 2023 @04:50PM (#35574565)
I would go by how well humans do it, which would mean: "yes, you can probably talk it out of it, but when it matters, that's hard enough to do in practice such that the human/system can be used for important tasks"
Re:Prompt injection: what’s the worst that can happen? by nebulousthree on Friday April 14, 2023 @05:02PM (#35574722)
With humans it's just that the stakes are high because you cannot generally have a human at your beck and call like we do with machines in general. If you limited to 5 questions per topic and and overall use time limit, you might see different input still.
Re:Prompt injection: what’s the worst that can happen? by schoen on Friday April 14, 2023 @05:31PM (#35575057)
Cf. "Help, I'm a prisoner in a fortune cookie factory!".

Apparently something like this really does happen, although it continues to be hard to tell whether any particular instance is real:

https://www.businessinsider.com/chinese-prisoners-sos-messag...