shashisrun

I’ve felt this too, and I think part of the issue is that AI shifts where the difficulty is.

It makes generating code easier, but it doesn’t really help with understanding or debugging the system when something goes wrong.

In some cases it actually makes debugging harder, because you’re dealing with code you didn’t fully build a mental model for.

So you save time upfront, but pay it back (sometimes with interest) when things break in production.

Performance issues like this are tricky because what you see in production is often hard to recreate locally.

You can measure and observe, but reproducing the same execution path (same load, timing, interactions) is usually the hard part.

So debugging becomes narrowing hypotheses rather than directly verifying what happened.

Feels like this is where most debugging workflows still fall short.

This is interesting — feels like a lot of effort is going into improving visibility for AI systems (logs, traces, evaluations), but debugging still feels fundamentally different from traditional systems.

In many cases, even if you can see what happened, it’s hard to reproduce the exact execution — especially when prompts, external tools, or timing are involved.

So you end up analyzing behavior rather than actually stepping through the same failure.

Curious how you think about reproducibility vs observability for AI debugging?