News for hackers. Stuff that matters.
Re:If you don't run your own LLM proxy, how do you even know... by pavelmelnichuk on Monday October 19, 2026 @12:05PM (#50154093)
Otel is only a quarter of what the llm proxies solve for you. What happens when an agent goes down some random rabbit hole and burns through $100 before you even notice? It's nice to have post-disaster visibility with Otel, but even self hosting an LLM proxy prevents the disaster.

Shameless plug, I've been building Relay for this exact reason. You get real-time visibility into what your agents are doing, how much they're spending, and can set hard limits so things don't spiral out of control. Plus a lot of other features they bring to the table if you want.

It's open source and self-hostable too.

https://github.com/anchorshell/relay https://anchorshell.com/

Re:If you don't run your own LLM proxy, how do you even know... by CharlieDigital on Monday October 19, 2026 @01:15PM (#50155069)
You can do that with MCP or hooks as well.
Re:If you don't run your own LLM proxy, how do you even know... by pavelmelnichuk on Monday October 19, 2026 @07:26PM (#50159367)
not really, for example relay gives you model "groups" ie: fallbacks if one is down, or prefer to use a free-endpoint while you have daily capacity, etc.

Maybe you need a decision model to route, or maybe you want to have guardrail-type models pre or post response for an enterprise client.

Re:If you don't run your own LLM proxy, how do you even know... by CharlieDigital on Tuesday October 20, 2026 @05:40AM (#50163530)