AI should only run as fast as we can catch up

Re:AI should only run as fast as we can catch up by jascha_eng on Monday December 08, 2025 @01:31PM (#46195869)

Verification is key, and the issue is that almost all AI generated code looks plausible so just reading the code is usually not enough. You need to build extremely good testing systems and actually run through the scenarios that you want to ensure work to be confident in the results. This can be preview deployments or other AI generated end to end tests that produce video output that you can watch or just a very good test suite with guard rails.

Without such automation and guard rails, AI generated code eventually becomes a burden on your team because you simply can't manually verify every scenario.

[ Parent | on HN ]

Re:AI should only run as fast as we can catch up by bigbuppo on Monday December 08, 2025 @01:50PM (#46196097)

And with any luck, they don't vibe code their tests that ultimately just return true;

[ Parent | on HN ]

Re:AI should only run as fast as we can catch up by yuedongze on Monday December 08, 2025 @01:35PM (#46195930)

indeed, i see verification debt outweighing tradition tech debt very very soon...

[ Parent | on HN ]

Re:AI should only run as fast as we can catch up by catigula on Monday December 08, 2025 @01:39PM (#46195963)

I can automatically generate suites of plausible tests using Claude Code.

If you can make as a rule "no AI for tests", then you can simply make the rule "no AI" or just learn to cope with it.

[ Parent | on HN ]

Re:AI should only run as fast as we can catch up by jopsen on Monday December 08, 2025 @06:07PM (#46198929)

I would rather write the code and have AI write the tests :)

And I have on occasion found it useful.

[ Parent | on HN ]