Should I reconsider my decision of self-hosting LLM Gateway?

Posted by leanroute_ai on Sunday September 06, 2026 @09:50AM
from the self-post dept.

The CVE-2026-35029 privilege escalation in LiteLLM is a reminder that middleware is not something you can install once and forget. What is your view on using self-hosted LLM gateway? I wrote this post - https://leanroute.dev/blog/self-hosting-an-llm-gateway Would like to hear your views.

( 4 comments | on HN )

Re:Should I reconsider my decision of self-hosting LLM Gateway? by FatalLogic on Sunday September 06, 2026 @10:29AM (#49586856)

Isn't this a general class of problem that has a similar risk profile for any type of gateway that accepts inputs that are not very tightly bounded?

Not only LLM services.

[ Parent | on HN ]

Re:Should I reconsider my decision of self-hosting LLM Gateway? by leanroute_ai on Wednesday September 16, 2026 @10:24AM (#49727509)

yes, but LiteLLM was exposed to a CVE. I switched to leanroute.dev

[ Parent | on HN ]

Re:Should I reconsider my decision of self-hosting LLM Gateway? by verdverm on Sunday September 06, 2026 @12:51PM (#49588353)

I'm running a gateway for my local LLMs, are you opening it to the internet? Who's going to craft a malicious payload?

[ Parent | on HN ]

Re:Should I reconsider my decision of self-hosting LLM Gateway? by leanroute_ai on Wednesday September 16, 2026 @10:26AM (#49727554)

was using LiteLLM for the company i work for. we were doing a poc.

But now we switched to leanroute.dev, we use our own provider keys. Gateway helps with the routing, caching, spend cap and monitoring usage.

[ Parent | on HN ]