News for hackers. Stuff that matters.
Re:Show HN: Vapi – Convince our voice AI to give you the sec... by lolinder on Thursday April 18, 2024 @11:50PM (#40083273)
You may already know this, but I thought it's worth clarifying in case someone gets the wrong idea: LLM models themselves cannot "get out of their EC2 instances". The code that runs inference on the models may be programmed to execute arbitrary code or use tools in response to special tokens. That tool-use code should be programmed and operated on the assumption that the LLM output is adversarial and sandbox the code executions accordingly.

It's the same idea as "don't inject user-provided strings directly into SQL queries". In every system you should keep track of the bits that are user input and treat that data as dangerous. The only difference with LLMs is that a lot of naive programmers forget that the LLM is itself untrusted.

Re:Show HN: Vapi – Convince our voice AI to give you the sec... by mlinhares on Friday April 19, 2024 @12:22AM (#40083406)
A lot of the expected usages for LLMs for "businesses" are about them making decisions, like agents, so I'm sure we will see multiple companies making the mistake of letting an LLM that has agent powers out in the wild and people will use these prompt hacks to get what they want.

Feels like a much simpler way of hacking systems.