Re:If you don't run your own LLM proxy, how do you even know...
by lokar on Monday October 19, 2026 @02:54PM
(#50156389)
IMO, a key place for proxies is resource access where the client is not fully trusted (or for any reason it may fail to log correctly).
AI agents seem like a pretty clear case of this. Both because of recent issues, and that they tend to run on end user devices where strict compliance is harder to maintain.
Standards based, many off-the-shelf providers, out-of-process + buffered by default, many standard processors for traces + spans, many tools for storing and querying.
OTEL is pretty widely supported and mostly "just works" at this stage of maturity. A failing/mis-configured OTEL endpoint/collector isn't going to stop you from getting work done in the same way that a failing proxy is. You'll drop some traces, but that's probably OK in most cases.
There is a place for proxies, but those places should share the same failure boundary mode. Meaning if the proxy fails, it should necessarily stop forward agent activity. Example: proxy injection of secrets instead of local acquisition.