Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by CuriouslyC on Sunday September 21, 2025 @05:58PM
(#45327052)
Kind of amazes me how many people bitch about agent performance but don't hook their guys up to Otel, crack Phoenix and get to work, but instead randomly tweak prompts in response to team vibes.
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by chrisweekly on Sunday September 21, 2025 @06:32PM
(#45327271)
Good point. Also (tangent), I followed your profile link to https://sibylline.dev and am thoroughly impressed. Stoked to have found your treasure trove of repos and insights.
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by CuriouslyC on Sunday September 21, 2025 @07:27PM
(#45327607)
Don't play with them unless you're good at debugging alpha code (claude/codex can do it fine), I haven't ironed out env specific stuff or clarified the installation/usage, and I'm still doing UI polish/optimization passes (yay async simd rust). I'll do showy releases once I've got the tools one click install ready, in the meantime please feel free to drop an issue on any of my projects if there are features or questions you have.
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by CuriouslyC on Sunday September 21, 2025 @11:23PM
(#45328811)
With Phoenix + Clickhouse being fed from Otel, you can do queries over your traces for deep analysis. If I want to see which tool calls are failing and why (or just get tool statistics), or find common patterns in flagged/failure traces ("simpler solution") and their causes, it's one query and some wiring.
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by CuriouslyC on Monday September 22, 2025 @02:40AM
(#45329830)
You can get traces out of every agent framework that doesn't suck. You might need to collect from a proxy to get them from Claude because Anthropic is bigtime amateur hour.
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by oefrha on Sunday September 21, 2025 @07:41PM
(#45327680)
Collecting detailed per-request traces and calculating user-specific metrics finer than a total cost feels about as intrusive as one of those periodic screenshot programs forced by really shitty remote jobs or freelancing contracts. It's pretty gross.
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by pranay01 on Sunday September 21, 2025 @09:31PM
(#45328267)
I don't think the primary goal here is "surveillance" but better understanding where in the team are tools like claude code getting adopted, what models are being used, are there best practices to learn in token usage which could make it more efficient
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by N_Lens on Monday September 22, 2025 @12:28AM
(#45329100)
I’d like to see this leveraged for agent platforms & orchestration rather than for surveillance on human software engineers. Humans don’t perform well in panopticons, but robots do (In my humble opinion).
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by N_Lens on Tuesday September 23, 2025 @01:12AM
(#45343047)
Claude Code Agents can be integrated into existing platforms such as github. I can envision agents automatically handling issues with certain tags, or doing pull request reviews, or other such similar trigger based behaviour.
In that kind of orchestration this observability would be invaluable.
Re:We instrumented Claude Code with OpenTelemetry (tokens, c...
by pranay01 on Tuesday September 23, 2025 @03:57PM
(#45351984)
interesting. So, you mean say if an agent is working on automatically doing a PR review, how many such calls to agents are failing, how much time they are taking, etc?
Lot of this you can do with traces today which trace AI specific calls
but the documentation only specifies metrics and logs
https://docs.claude.com/en/docs/claude-code/monitoring-usage
am i missing something here? or is this a case where they were lazy to do traces in the claude code sdk, so its logs + log attributes?
can you share more on what you mean by this?
In that kind of orchestration this observability would be invaluable.
Lot of this you can do with traces today which trace AI specific calls