What I am most interested in is the gap between "the agent completed the task" and "the system can actually prove the task was completed correctly." That is where a lot of multi agent setups still feel fragile to me.
LLM review definitely helps, but I think it gets much stronger when it is paired with deterministic checks in the loop, even simple ones like executable smoke tests, schema validation, contract checks or replayable fixtures. Otherwise you can end up with persuasive agreement rather than real verification.
This framing...
> Otherwise you can end up with persuasive agreement rather than real verification.
... is very familiar and one of the inspirational elements that led to https://github.com/spencermarx/open-code-review/tree/main (not trying to shamelessly plug, just your articulation was right on the money).
It's definitely the holistic set of static analysis and a robust automated test pyramid, then agentic analysis (ideally differentiated, and redundant with discourse -- like Open Code Review), that is proving to be effective at ensuring quality code.
That said, and a bit of a premature callout to the other project I mentioned, I think structured knowledge graphs to describe and commit to software intent (beyond what nested AGENT/CLAUDE/rule files and free form docs give you) are going to help us do a better job of engineering context and prompts for agentic development. This is the area that still feels like the wild west. Standardization is helping, and I think the toolset available now between RAG, lifecycle hooks, progressive disclosure in context and prompts, plus multi-agent implementation, review and discourse will help to close the loop and get us to the point of really confident and consistent pure AI-driven development. That's just my POV lol, I'll hop off my soap box now haha
If you're looking to see what that feedback mechanism might look like in action, you might like checking out one of the other projects I've worked on which pre-dates Orc: https://github.com/spencermarx/open-code-review
Love where your head is at though! DEFINITELY an important problem we've got to solve here.