News for hackers. Stuff that matters.

A comment on When AI Writes the Software, Who Verifies It?. Parent comment

Re:When AI Writes the Software, Who Verifies It? by MartyMcBot on Tuesday March 03, 2026 @10:55PM (#47242858)
the test generation loop is the real trap. you ask the agent to write code, then ask it to write tests for that code. of course the tests pass. they're testing what the code does, not what it should do.

we ran into this building a task manager. the PUT endpoint set completed=true but never set the completion timestamp. the agent-written tests all passed because they tested "does it set completed to true" not "does it record when it was completed." 59 tasks in production with null timestamps before a downstream report caught it.

the fix was trivial. the gap in verification wasn't.

[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by tbossanova on Tuesday March 03, 2026 @11:31PM (#47243073)
Once upon a time people advocated writing tests first…
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by codegladiator on Tuesday March 03, 2026 @11:51PM (#47243225)
once upon a time 'engineering' in software had some meaning attached to it...

no other engineering profession would accept the standards(or rather their lack of) on which software engineering is running.

[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by hulitu on Wednesday March 04, 2026 @03:02AM (#47244494)
> no other engineering profession would accept the standards(or rather their lack of) on which software engineering is running.

I have bad news for you: they are pushing those "standards" (Agile, ASPICE) also in hardware and mechanical engineering.

The results can be seen already. Testing is expensive and this is the field where most savings can be implemented.

[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by philipallstar on Wednesday March 04, 2026 @07:39AM (#47246621)
Agile isn't a coding standard or approach.
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by 8note on Wednesday March 04, 2026 @12:45AM (#47243561)
i dont think that would help. the agent would hard code the test details into the code.
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by tbossanova on Friday March 06, 2026 @12:28AM (#47271220)
Ah, so another way they’re like humans haha
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by scotty79 on Wednesday March 04, 2026 @02:24AM (#47244279)
I wasn't able to force the agent to write failing tests yet. Although I'm sure it should be possible to do.
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by himlion on Wednesday March 04, 2026 @02:44AM (#47244383)
I do that all the time with Claude. What part is not working?
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by scotty79 on Wednesday March 04, 2026 @04:29AM (#47245105)
I don't really use anthropic models. But when I tried it with others they can write tests but they never confirm that they fail before they proceed to make implementation that causes them to pass. Maybe I didn't prompt it forcibly enough.
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by catlifeonmars on Wednesday March 04, 2026 @07:39AM (#47246629)
I haven’t tried this (yet), but I’ve heard of people disabling write access to test code while the agent is writing implementation and vice versa. I imagine “disabling” could be done via prompting, or just a quick one liner like: chmod -r 0644 ./tests
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by eithed on Wednesday March 04, 2026 @06:10AM (#47245894)
Once upon a time people were thinking about what they're doing. LLMs absolve people from thinking
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by harimau777 on Wednesday March 04, 2026 @06:38AM (#47246080)
Engineers aren't paid to think. They are paid to be replacable cogs who can be fired the moment they show independent thought.
[ Parent | on HN ]
Re:When AI Writes the Software, Who Verifies It? by theshrike79 on Wednesday March 04, 2026 @08:30AM (#47247139)
The magic word is "use red/green testing", that makes it create the tests first, confirm they fail (as they should), then it writes the code to match.
[ Parent | on HN ]