News for hackers. Stuff that matters.
Re:Show HN: You don't need to adopt new tools for LLM observ... by hooverd on Wednesday February 14, 2024 @03:23PM (#39374790)
It's LLM specific OpenTelemetry tracing. What's going on inside your model isn't the focus. It's everything surrounding your model. How many prompts are people submitting? How long does each prompt take? Did certain prompts time out or return an error? What's the P95/P99 latency for your LLM? And so on.

There are no comments on this comment yet.