I'm saying we need to move away from "LGTM" focused PR reviews. Most companies have a PR review focused on getting thumbs-up emojis in the comment, then they merge and have a sprint planning. Nobody ever reviews the entire system. Nobody ever asks "assuming there is a mistake in this PR, would it matter? How can we make it so that it wouldn't matter?"
I never claimed AI makes no mistakes. People makes mistakes, and reviewers make mistakes too.
I've worked at a lot of mediocre companies where the plan to keep the system working is "the developer will not make a mistake, and the code reviewer will not make a mistake, this is our plan, our plan is humans will not make mistakes, and then our system will continue working".
It's better to ask "let's assume a human (or an AI) does make a mistake, how can we design the system to prevent that from mattering?"
Most companies don't have the capacity to do that though, so they never move beyond thumbs-up emojis.
you can't
there is no way around it
we need far more review than we economically are motivated to sustain
and hence issues like we've been having over the past week will continue and perhaps worsen
software will (as it already has been) get crappier and crappier overtime
Humans can be ultra competent if people are actually paying them to be artisans or otherwise motivating them to truly be that way. That's how we have some of the architecture we have today and historically.
This is a No True Scotsman argument.
The prescription you suggest is not dependent on its inspiration, so it can still be discussed.
All firms right now, even those with better than average cultures, are dealing with AI code workflows.
Previously the volume of code generated over time was lower, even mediocre code review was useful enough over all the code that people generated.
Today, the ratio of verification capacity to generation capacity has shifted drastically. Even with automated tooling, the amount of verification isn’t enough to stay on top of the generated code.
This changes only when leadership values reviewer capacity, or builds a triage/priority list for code which must be understood.
> I never claimed AI makes no mistakes. People make mistakes, and reviewers make mistakes too.
These are 2 separate things. The crux of the issue is the push towards utility over understanding. The volume of generated code means that it is impossible for understanding about the system to be achieved - and it is now the universal coding experience, not just mediocre firms.
Quantity is its own quality, as well so even a good culture will struggle to stay on top of review.
Each function declares what it can touch (e.g. network, filesystem, clock) and the compiler and runtime work together to enforce it. However, I have yet to show it beats LLM-written Python plus tests and a sandbox. Models have gotten good enough at writing code that I haven't found much room for enforcement to help.
This requires continuous commitment against the temptation to defer "regular maintenance". Look at how regular maintenance is mandated in safety critical fields, even there there is a temptation to defer. Though, your approach might still be better than sham reviews.
The way you would naturally review an existing codebase is not the same way it grows through individual commits. Therefore, focusing reviews on individual commits is less efficient.
We need better tooling and a better approach.