News for hackers. Stuff that matters.

A comment on Show HN: I built an analytics tool to track cost and usage of any LLM provider. Parent comment

Re:Show HN: I built an analytics tool to track cost and usag... by verdverm on Saturday May 18, 2024 @09:59PM (#40403577)
I don't use OpenAI because it is an unreliable service, they don't know how to run production.

LangFuse supports all providers, allows for user and session level metrics, and is self hostable. Custom spans are really helpful as is the ability to take feedback on responses and build data sets from them.

How do you include your RAG queries and non-ai ops into your overall chat observability?

I'm somewhat of a mind to switch to an Open Telemetry based system, since that is the standard now

[ Parent | on HN ]
Re:Show HN: I built an analytics tool to track cost and usag... by kakaly0403 on Saturday May 18, 2024 @10:14PM (#40403634)
Hey I am building an open telemetry based open source tool for metrics and logs from LLM based apps. Give it a try at langtrace.ai
[ Parent | on HN ]
Re:Show HN: I built an analytics tool to track cost and usag... by verdverm on Saturday May 18, 2024 @10:41PM (#40403779)
Will do, thanks for posting

One question I would have is why use a custom UI and storage setup, rather than the existing otel setup for the rest of the infra?

i.e. how much is being reinvented or duplicated from my grafana stack?

[ Parent | on HN ]