skhatter

Working on multi-step agent systems (agents calling tools, other agents, APIs), I keep running into a frustrating class of failures:

One step returns slightly malformed or incomplete state Downstream steps continue executing anyway The issue only surfaces several steps later

Nothing actually “fails” — the system just produces the wrong result.

These are hard to catch:

Logs/traces help explain what happened after the fact But they don’t prevent bad execution from propagating

I’m experimenting with:

Explicit state validation between steps Blocking unsafe transitions Replay from intermediate failure points

Curious how others are handling this in production. Are you relying purely on tracing/logs, or enforcing stricter contracts between steps?

I’m building something in this space and looking for a few design partners to try it out (happy to wire it up myself): https://www.agentsentinelai.com/

That makes sense — sounds like a lot of this is handled at the framework + design level in your setup.

In practice, when something does go wrong in a multi-step workflow, do you typically rely on tracing + manual debugging, or do you have built-in mechanisms for partial replay / recovery?

The “authority boundary” framing is really helpful — tracing explains what happened, but not whether a transition between agents should have been allowed.

Curious how teams are handling this today — are those contracts usually defined explicitly (schemas / validators), or are they mostly implicit in the agent code and discovered only after failures?

This is a great example — feels very similar to classic lost update problems in distributed systems. The propose/validate/commit cycle makes a lot of sense.

Curious how you're handling this in practice — are all shared state mutations going through that flow, or only critical paths? And does the coordination overhead become a bottleneck as workflows scale?

Interesting — are you instrumenting the agent workflows themselves with OpenTelemetry spans?

I was wondering how well the standard o11y stack works once agents start running multi-step workflows (agent → tools → other agents → APIs). Tracing probably helps visualize the steps, but I'm curious how people handle operational things like retries, replaying failed workflows, or containing cascading failures across agents.

Those reliability aspects are what I've been exploring.

I've been experimenting with AI agents and multi-step workflows recently and ran into a problem that reminded me a lot of early distributed systems.

Once agents start calling tools, APIs, and other agents in a chain, debugging failures becomes surprisingly hard. A single task can involve multiple steps—LLM calls, tool invocations, retries—and when something breaks it's often difficult to understand exactly what happened or where the failure originated.

In traditional distributed systems we eventually built things like tracing, circuit breakers, retry policies, SLOs, and other reliability primitives to operate systems safely in production.

I'm curious how people building agent systems today are handling this.

Some questions I'm particularly interested in: - How do you debug agent failures? - Do you have visibility into multi-agent workflows? - How do you replay or reproduce failures?

I've been exploring this problem space and built a small prototype to experiment with reliability tooling for agent systems. The link above shows the demo, but I'm mainly interested in learning how others are approaching this problem.