Re:How the Claude Code team thinks about verification
by _nalply on Tuesday November 17, 2026 @08:47AM
(#50531319)
Verification? That's not so simple. Essentially humans must do it, or do it with an agent on a tight leash.
When working with an agent, I encountered two failure modes.
1. Automated testing. Once the agent proposed a complicated oracle. Because I didn't understand it, I thought, okay, whatever. But the oracle was wrong.
2. Reporting. The agent lied while writing the report. Once the agent said it's doing "exec app args", but it was "exec /bin/sh -c helper-script app args". That's just not the same.
Generally it amounts to "Quis custodiet ipsos custodes?" - this means verification is difficult, because you need to verify the verification. Even if you use a second agent to verify the verification, you don't know whether the second agent is doing what you mean.
Do what I mean is slippery with agents.
Currently I solve this problem by careful and strict with rules and asking many questions and if I don't understand something, I assume that the agent might have done something different than I expect.
When working with an agent, I encountered two failure modes.
1. Automated testing. Once the agent proposed a complicated oracle. Because I didn't understand it, I thought, okay, whatever. But the oracle was wrong.
2. Reporting. The agent lied while writing the report. Once the agent said it's doing "exec app args", but it was "exec /bin/sh -c helper-script app args". That's just not the same.
Generally it amounts to "Quis custodiet ipsos custodes?" - this means verification is difficult, because you need to verify the verification. Even if you use a second agent to verify the verification, you don't know whether the second agent is doing what you mean.
Do what I mean is slippery with agents.
Currently I solve this problem by careful and strict with rules and asking many questions and if I don't understand something, I assume that the agent might have done something different than I expect.