Re:Show HN: Vapi – Convince our voice AI to give you the sec...
by mlinhares on Friday April 19, 2024 @12:22AM
(#40083406)
A lot of the expected usages for LLMs for "businesses" are about them making decisions, like agents, so I'm sure we will see multiple companies making the mistake of letting an LLM that has agent powers out in the wild and people will use these prompt hacks to get what they want.
Feels like a much simpler way of hacking systems.
It's the same idea as "don't inject user-provided strings directly into SQL queries". In every system you should keep track of the bits that are user input and treat that data as dangerous. The only difference with LLMs is that a lot of naive programmers forget that the LLM is itself untrusted.