This is how we use it for code reviews:
- a skill tells the agent to automatically run a subset of tests and linting before each commit
- another skill tells it to review the entire changeset before creating a PR, this review has more extensive rules that can't easily be put into code (e.g. linter rules) based on PR comments humans have written. It also sometimes catches things that were missed from the original prompt/task.
- when the PR is created we run a few AI tools to do automated code and security reviews. CI runs at the same time.
- the agent waits for these to complete, and verifies and fixes any issues if they are valid
- after all that it's passed back to the author to review
- once they are happy it's passed to a teammate to review
So we are not handing off reviews to AI, we are using it to do much more extensive reviews, and automatically fix stupid stuff the AI or human might have done. So by the time you are asked to review a PR, it should be pretty much ready to go, you can focus on what it's actually changing instead of looking for slop.
Did you miss or already forget the context of "humans no longer needing to understand a codebase, and letting AI drive it"? You're not doing that, either. You cannot "review" something you don't understand. You can "try it out" maybe.
I wouldn't blame your "lazy" prompting. Specification is just really hard. This is why we stopped doing waterfall software development. I think the current-day obsession with one-shotting software forgets why we had to stop trying to figure everything out up front.
Hard disagree. Even the best frontier models generate output that's not what I asked for. Sometimes I realize that I get lazy in my prompting and the lack of specificity winds up showing up in the output. Just the other day, a coworker built a huge feature using frontier models and it slipped an IDOR in.
I just don't see a world in which we completely cede control of the codebase to AI because it's still my ass on the line if I ship something that completely borks production. If I'm not reading code regularly, then I lose the ability to read code, and if I lose that ability, then I'm no longer a developer.