Re:Show HN: You don't need to adopt new tools for LLM observ...
by a_wild_dandan on Wednesday February 14, 2024 @01:28PM
(#39373315)
What problem(s) does this solve? I have a ticket in my backlog. Your SDK unlocks the solution. What is that ticket's title? (I'm a bit thick, and need concrete examples for things to click.)
[ Parent
| on HN ]
Re:Show HN: You don't need to adopt new tools for LLM observ...
by nirga on Wednesday February 14, 2024 @01:57PM
(#39373658)
Same ticket that gets you to install something like Sentry - you wanna see what's happening in production and get alerted when things go wrong
Re:Show HN: You don't need to adopt new tools for LLM observ...
by hooverd on Wednesday February 14, 2024 @03:23PM
(#39374790)
It's LLM specific OpenTelemetry tracing. What's going on inside your model isn't the focus. It's everything surrounding your model. How many prompts are people submitting? How long does each prompt take? Did certain prompts time out or return an error? What's the P95/P99 latency for your LLM? And so on.