News for hackers. Stuff that matters.
Re:Delimiters won’t save you from prompt injection by numpad0 on Saturday May 13, 2023 @02:50PM (#35931434)
If that's possible, will it be also possible to characterize/model how parameters dissolve into a weight and "forward-pass" analytically construct LLM/DNN models?
Re:Delimiters won’t save you from prompt injection by PeterisP on Saturday May 13, 2023 @04:42PM (#35932491)
I'm not sure how those things would be related.

The above post is about ensuring that the markings given to the model along with the text about the prompt/data distinction are "out-of-band", reliable, and can't be influenced or faked by user-controlled data. Having the model actually act in accordance to the prompt is a wholly different issue; but at least this discussion seems to assume that this is mostly solved (e.g. by reinforcement learning from human feedback) and that the main problem is the injection itself.