Compromising LLM-Integrated Applications with Indirect Prompt Injection

Re:Compromising LLM-Integrated Applications with Indirect Pr... by alangpierce on Monday May 22, 2023 @08:09PM (#36038309)

This article argues that there's no reliable way to detect prompt injection: https://simonwillison.net/2022/Sep/17/prompt-injection-more-...

One solution to some indirect prompt injection attacks is proposed in this article, where you "sandbox" untrusted content into a second LLM that isn't given the ability to decide which actions to take: https://simonwillison.net/2023/Apr/25/dual-llm-pattern/

[ Parent | on HN ]

Re:Compromising LLM-Integrated Applications with Indirect Pr... by SkyPuncher on Monday May 22, 2023 @09:09PM (#36038636)

I see absolutely no way prompt injection can be fully protected against.

There are nearly infinite ways to word an attack. You can only protect against the most common of them.

[ Parent | on HN ]

Re:Compromising LLM-Integrated Applications with Indirect Pr... by cubefox on Monday May 22, 2023 @09:38PM (#36038799)

What about this approach?

https://news.ycombinator.com/item?id=35929145

[ Parent | on HN ]

Re:Compromising LLM-Integrated Applications with Indirect Pr... by SkyPuncher on Monday May 22, 2023 @11:57PM (#36039647)

I mean, sure that'd work, but doesn't it defeat most of the point in using an LLM?

The only way that works is if you escape _all_ user content. If you're telling an LLM to ignore all user content, then why are you using an LLM in the first place?

[ Parent | on HN ]

Re:Compromising LLM-Integrated Applications with Indirect Pr... by cubefox on Tuesday May 23, 2023 @04:12AM (#36041408)

The approach isn't to ignore all "user" content at all. It is trained to follow instructions in normal text; only instructions contained in specially quoted text (that is, external text, like a website) are ignored. Quotation would apply to Bing's search abilities or ChatGPTs new Browsing Mode, which both load website content into the context window.

[ Parent | on HN ]