If you don't run your own LLM proxy, how do you even know whats going on?

Re:If you don't run your own LLM proxy, how do you even know... by zaptheimpaler on Monday October 19, 2026 @01:10AM (#50148594)

I agree its so important to proxy and capture all the data. I've been very happy with bifrost as a proxy. It's a quality piece of software, unlike LiteLLM. You also get this one interface to access all your models, across all services, including self-hosted, STT models, everything. There are ways to use subscriptions with proxies too, but they are discouraged because of grey-hat uses..

[ Parent | on HN ]

Re:If you don't run your own LLM proxy, how do you even know... by hdra on Monday October 19, 2026 @03:14AM (#50149327)

prob a noob question; but what kind of data are people looking to get to help with what kind of decisions with these tools?

i can see some IT/infra team using these tools to manage their LLM costs, but this blog post seems to hint at some kind of product decision making? what am i missing here?

[ Parent | on HN ]

Re:If you don't run your own LLM proxy, how do you even know... by abhikap on Monday October 19, 2026 @04:35PM (#50157700)

Currently using litellm and its been pretty solid for me (have my claude sub on there as well) - curious why bifrost over litellm? Also iiuc subs on proxies is ok if you're not farming them.

[ Parent | on HN ]

Re:If you don't run your own LLM proxy, how do you even know... by zaptheimpaler on Wednesday October 21, 2026 @03:15PM (#50186063)

It's funny another massive vulnerability in LiteLLM is on top of HN now. They had another big vulnerability earlier too. If you look at the history of the project and the design decisions, I believe LiteLLM is slop made by bad programmers and I would stay far away.

[ Parent | on HN ]