There will also be models that make better use of tools, like static analysis and fuzzing, and they will find different defects. Social engineering is going to be a big skill to learn too, as it is a much softer skill.
Say more? HTTP isn't mentioned in the article. The article talks about bugs in the parser that ingests token sequences.
e.g. for a vulnerability exploit in the parser, an LLM can be trained with a specific input sequence that would output a fixed malicious payload to output eval-able content, as outlined in the article.
Vulnerable code could exist in token generation and in hook recognition and tool call parsing [1]. There is also significant scope for mischief in tool ID mapping between models and harnesses, as these are done by untyped numeric IDs, with varying schema[2]. Routers also introduce vulnerability paths as they inspect these tokenized (json) sequences and act in them, e.g. to match models with stricter call signature regex. [3] Parallel tool calling is also an interesting surface for exploits.
[1] https://docs.vllm.ai/en/stable/api/vllm/tool_parsers/#vllm.t...
[2] https://docs.mistral.ai/resources/cookbooks/concept-deep-div...
Basically, it's a regex, don't fuck it up.
which is exposed via http
I found it lacking details. All these things do is split the input into tokens, encode them, feed to the input, run the inference, collect the output, decode, convert to tokens, assemble the final text, and output (leaving aside multi-modal capabilities for now). What exactly can be exploited here? Aside from usual things like buffer overruns etc.
It does mention that vLLM at some point ran eval() on final output (?) which was confusing, why would it do that? It's an inference engine, not an agent.
So interesting topic, but lacks details.
Edit: many (all?) of them have http endpoints, so that obviously can be exploited, but I don't think it would qualify as exploiting inference, it's just hacking the http service.
There is lots of surface inside the engine, see links in https://news.ycombinator.com/item?id=49441417
There's a couple of different conventions for what the LLM generates for tool calls, I think the code in vLLM is converting it from whatever Qwen3 was trained for to whatever convention the HTTP API wants to expose.
That part about using eval in 2025 got me to add "#naive" to my notes about vLLM. Total WTF. This should never have been done.
https://github.com/vllm-project/vllm/pull/21396#discussion_r...
vLLM has had exploits in the past, and it is rapidly developing. An advanced LLM has a good chance of being able to exploit vLLM. A clever local LLM might even task a powerful cloud hosted LLM for assistance.
For this reason we run vLLM on a separately sandboxed VM on a firewalled VLAN. Software updates and models (from Dev/Test env) get pushed onto Prod from an external cache, machine syslog, nvidia load monitoring and vLLM query telemetry out to their loggers, but that is all. No DNS, no AD/LDAP, nothing. Firewall on hosts and VM hosts. Log and telemetry processing done on a completely separate set of VMs in their own isolated subnet, producing reports and alerts that are tightly formatted.