ianmcgraw

OpenTelemetry has two competing semantic conventions for tracing agent behavior: the OTEL GenAI standard and OpenInference. We've found OpenInference to be significantly more useful for production agent workloads, offering richer LLM call metadata, first-class support for RAG and retrieval spans, and better distinctions between span types. We've found that projects that use OpenInference vs. OTEL GenAI standard are better positioned to improve individual components of their agentic architecture (eg: Prompts, Tool Calls, RAG, etc.), allowing them to ship faster and with more confidence. This post covers why, along with what to instrument and which frameworks give you strong observability out of the box.

Hey HN,

Lead engineer at Arthur AI here. For the last year, my team and I have been in the trenches with customers (in finance, airlines, etc. ) trying to get AI agents from a cool demo to a reliable product.

The problem we hit over and over is that Agentic AI is easy to get to a functionally complete state, but going from functionally complete to reliable is where most teams struggle.

We found that the traditional SDLC process doesn’t work for agentic systems since these systems are probabilistic. Agent development requires iteration and experimentation to align behavior with business objectives.

We needed a new methodology! Today, we're sharing the one we've developed and refined after putting this into practice: The Agent Development Lifecycle (ADLC).

The core of the ADLC is a shift from the linear SDLC to a continuous loop we call the Agent Development Flywheel. This flywheel allows us to methodically identify failure modes from live and simulated usage and add them to an evolving evaluation behavior suite. This suite then allows us to confidently experiment with new prompts or tools to improve the agent's performance without introducing new regressions.

I’d love to hear what you think. I'm here to answer any questions about what we've seen work (and not work) in production.

Yes that's correct. Argo requires Kubernetes so it's actually impossible to use without it. Argo is great for orchestrating containers but falls short if you need to interact with anything else.