News for hackers. Stuff that matters.
Re:Compromising LLM-Integrated Applications with Indirect Pr... by cubefox on Monday May 22, 2023 @09:38PM (#36038799)
Re:Compromising LLM-Integrated Applications with Indirect Pr... by SkyPuncher on Monday May 22, 2023 @11:57PM (#36039647)
I mean, sure that'd work, but doesn't it defeat most of the point in using an LLM?

The only way that works is if you escape _all_ user content. If you're telling an LLM to ignore all user content, then why are you using an LLM in the first place?

Re:Compromising LLM-Integrated Applications with Indirect Pr... by cubefox on Tuesday May 23, 2023 @04:12AM (#36041408)
The approach isn't to ignore all "user" content at all. It is trained to follow instructions in normal text; only instructions contained in specially quoted text (that is, external text, like a website) are ignored. Quotation would apply to Bing's search abilities or ChatGPTs new Browsing Mode, which both load website content into the context window.