Re:Show HN: You don't need to adopt new tools for LLM observ...
by tomerf2 on Wednesday February 14, 2024 @11:48AM
(#39372002)
Thanks!
1. Huh, good question. Hopefully there will be convergence. We started discussing with other companies in this domain to support or even switch to OpenTelemetry.
2. Nothing specific, except for - as you mentioned - being able to see trace of a RAG pipeline automatically.
Re:Show HN: You don't need to adopt new tools for LLM observ...
by tomgs on Wednesday February 14, 2024 @11:54AM
(#39372073)
A quick one :)
While we're on the topic - how does traceloop factor into all of this? What's the connection between the two? I assume the former is the LLM observability platform (Datadog for LLM?) and the latter is your own auto-instrumentation thingie to supplement it?
Re:Show HN: You don't need to adopt new tools for LLM observ...
by nirga on Wednesday February 14, 2024 @11:58AM
(#39372130)
Yes, Traceloop is kind of a Sentry for LLMs
1. Where do you see this observability for LLM thing going? What's the end game? Is it like in traditional observability where all formats eventually will converge to one format (which OpenTelemetry is trying to be)? I feel it might be a little bit early to tell, tho
2. I noticed you do auto-detection of the framework used, like LLamaIndex et al. Except for annotations, is there a deeper connection to the LLM framework used? This is auto-instrumentation, so I assume you do most of the heavy lifting, but should users of this framework expect some cool hidden eggs when they look at their telemetry?