That said, I have a hunch we're heading toward a world where we stop reading AI-generated code the same way we stopped reading assembly. Not today, not tomorrow, but the direction feels clear.
Until then — yes, we need to understand every bit of what the AI writes.
AI? Not so much. Not deterministic. Sure, the probability of something bizarre may go down. But with AI, as currently constituted, you will always need to review what it does.
The real comparison is: 1. Human writes code (non-deterministic, buggy) → compiler catches errors
2. AI writes code (non-deterministic, buggy) → compiler catches errors
In both cases, the author is non-deterministic. We never trusted human-written code without review and compilation either (and + lots of tests). The question isn't whether AI output needs verification — of course it does. The question is whether AI + human review produces better results faster than human alone.
I’ve been actively trying to apply AI to our field, but the friction is real. We require determinism, whereas AI fundamentally operates on probability.
The issue is the Pareto Principle in overdrive: AI gets you to 90% instantly, but in our environment, anything less than 100% is often a failure. Bridging that final 10% reliability gap is the real challenge.
Still, I view total replacement as inevitable. We are currently in a transition period where our job is to rigorously experiment and figure out how to safely cross that gap.
Good luck!