It started originally as a way to limit costs (the proxy would intercept requests, estimate the token sizes, and block requests before they are sent to OpenAI). However, at the request of some early users, I’ve expanded it to include things like keyword detection/blocking, moderation enforcement, etc.
I’m not entirely convinced you can ever fully block prompt attacks, but right now most companies are just asking for visibility into it. So you could monitor for things like: do certain malicious phrases appear in the request? Or does a significant percentage of the original prompt text also appear in the response (a signal that the prompt is leaking).
It started originally as a way to limit costs (the proxy would intercept requests, estimate the token sizes, and block requests before they are sent to OpenAI). However, at the request of some early users, I’ve expanded it to include things like keyword detection/blocking, moderation enforcement, etc.
I’m not entirely convinced you can ever fully block prompt attacks, but right now most companies are just asking for visibility into it. So you could monitor for things like: do certain malicious phrases appear in the request? Or does a significant percentage of the original prompt text also appear in the response (a signal that the prompt is leaking).