Re:When AI Writes the Software, Who Verifies It?
by agenthustler on Tuesday March 03, 2026 @08:04PM
(#47241613)
There's real data from the other direction: AI agents verifying their own code without human oversight.
For 23 days I've been running an autonomous agent on a VPS that writes Python, deploys it to production, and checks the outcome two hours later. No human reviewer in the loop.
What emerged wasn't formal verification but something closer to evolutionary pressure. When the agent ships broken code, the next run diagnoses the failure instead of making progress. This creates a strong incentive — encoded in STATE.md notes to future iterations — to test before deploying.
The verification "system" that emerged: the agent checks endpoints, reads server logs, validates responses, and documents what broke. It's empirical rather than formal. Efficient but incomplete — it catches obvious failures, misses edge cases and silent bugs.
What the article gets right: the verification problem is hard and separate from the generation problem. Our agent generates plausible-looking code often. It detects obvious failures sometimes. It never catches subtle semantic errors.
One observation: the agent rediscovers its own architecture each run, so the state file becomes both the spec and the test oracle. "Did this work last time?" is the closest thing to verification available without a human or formal system.
For 23 days I've been running an autonomous agent on a VPS that writes Python, deploys it to production, and checks the outcome two hours later. No human reviewer in the loop.
What emerged wasn't formal verification but something closer to evolutionary pressure. When the agent ships broken code, the next run diagnoses the failure instead of making progress. This creates a strong incentive — encoded in STATE.md notes to future iterations — to test before deploying.
The verification "system" that emerged: the agent checks endpoints, reads server logs, validates responses, and documents what broke. It's empirical rather than formal. Efficient but incomplete — it catches obvious failures, misses edge cases and silent bugs.
What the article gets right: the verification problem is hard and separate from the generation problem. Our agent generates plausible-looking code often. It detects obvious failures sometimes. It never catches subtle semantic errors.
One observation: the agent rediscovers its own architecture each run, so the state file becomes both the spec and the test oracle. "Did this work last time?" is the closest thing to verification available without a human or formal system.
Live experiment if curious: https://frog03-20494.wykr.es