News for hackers. Stuff that matters.
Re:Design Patterns for Securing LLM Agents Against Prompt In... by simonw on Friday June 13, 2025 @11:46AM (#44269594)
If only it were as easy as that!

The problem with prompt injection is that the attack itself is the same as SQL injection - concatenation trusted and untrusted strings together - mint so far all of our attempts at implementing a solution similar to parameterized queries (such as system prompts and prompt delimiters) have failed.

Re:Design Patterns for Securing LLM Agents Against Prompt In... by Terr_ on Friday June 13, 2025 @01:17PM (#44270353)
Even worse, all outputs become inputs, at least in the most interesting use-cases. So to continue the SQL analogy, you can be 100% confident that in the legitimacy of:

    SELECT messages.content FROM messages WHERE id = 123;
Yet the system is in danger anyway, because that cell happens to be a string of:

    DROP TABLE customers;--
... Which becomes appended to the giant pile-of-inputs.

_____

Long ago I encountered a predecessor's "web scripting language" product... it worked based on repeatedly evaluating a string and substituting the result, until it stopped mutating. Injection was its lifeblood, Even an if-else was really just a decision between one string to print and one string to discard.

As much as it horrified me, in retrospect it was still marginally more secure than an LLM, because at least it had definite (if ultimately unworkable) rules for matching/escaping things, instead of statistical suggestions.