Delimiters won’t save you from prompt injection

Re:Delimiters won’t save you from prompt injection by robga on Saturday May 13, 2023 @04:30AM (#35926650)

I anticipate we’ll shortly have PAFs, “Prompt Application Firewalls”, on the market that externalise some of the detection and prevention from model publishers and act as network barriers in front of applications. Don’t leave it to model makers just as you don’t leave SQL injection prevention to developers alone. Not an easy task but it seems tractable. Unsolved, but soluble.

Zero Google results for the term. Perhaps there is another term and they already exist, eg baked into next gen WAFs.

[ Parent | on HN ]

Re:Delimiters won’t save you from prompt injection by instance on Saturday May 13, 2023 @04:44AM (#35926711)

I've put up something like this as a side project a few weeks ago: https://flamingoo.ai/

Love the term "PAF", I think I'll include this.

[ Parent | on HN ]

Re:Delimiters won’t save you from prompt injection by awestroke on Saturday May 13, 2023 @07:37AM (#35927418)

The website does not mention what the product is

[ Parent | on HN ]

Re:Delimiters won’t save you from prompt injection by instance on Saturday May 13, 2023 @02:07PM (#35930986)

yea, you're right. It's really pre-MVP. Basically an API in between your user facing input and OpenAI that detects prompt injections and flags them for you so you can abort sending to OpenAI.

[ Parent | on HN ]

Re:Delimiters won’t save you from prompt injection by cddotdotslash on Saturday May 13, 2023 @07:55AM (#35927504)

I’ve started working on something like this as a side project: https://usagepanda.com

It started originally as a way to limit costs (the proxy would intercept requests, estimate the token sizes, and block requests before they are sent to OpenAI). However, at the request of some early users, I’ve expanded it to include things like keyword detection/blocking, moderation enforcement, etc.

I’m not entirely convinced you can ever fully block prompt attacks, but right now most companies are just asking for visibility into it. So you could monitor for things like: do certain malicious phrases appear in the request? Or does a significant percentage of the original prompt text also appear in the response (a signal that the prompt is leaking).

[ Parent | on HN ]

Re:Delimiters won’t save you from prompt injection by quickthrower2 on Saturday May 13, 2023 @04:12PM (#35932238)

I believe they will exist, but I don’t think they will be effective at stopping the threat, but a good money making opportunity for someone who wants to sell the feeling of reassurance.

[ Parent | on HN ]