News for hackers. Stuff that matters.
Re:AI coding agents generate more code, but not more software by paimapi on Monday October 12, 2026 @04:01PM (#50062158)
And test creation and maintenance. AI-generated tests will often embed existing bugs as testable patterns that fail when fixed or rely on empty asserts to increase coverage. A human author generally knows better. Coverage is increasingly becoming theater and runtime testing eating up CI resources and time with bloated, unnecessary, untargeted testing that hasn't been mutation tested or had test oracles implemented are another drag.

It's really the smaller firms which are able to learn quick adaptations that will win out in the long-run of this thing. Legacy firms with institutionalized bureaucracies are relying on gobbling up competitors and creating vertical monopolies while pushing out the worst slop - it's why GH goes down, why Meta products are buggy as hell now, and everything feels like it's unpolished MVPs these days.

Re:AI coding agents generate more code, but not more software by btown on Monday October 12, 2026 @05:02PM (#50063025)
This. The ability to know that a vibe coded PR is all right to merge, is a function of both the thoughtfulness and simplicity of the underlying architecture, and the level of ownership that contributors have of the underlying products. Both benefit more nimble teams.
Re:AI coding agents generate more code, but not more software by bunderbunder on Monday October 12, 2026 @06:44PM (#50064118)
So far this year, of the defects severe enough to warrant a formal RCA process that I’ve dealt with, “AI generated tests asserted that the defective behavior was correct,” was a contributing factor in every single one.
Re:AI coding agents generate more code, but not more software by desolate_muffin on Monday October 12, 2026 @08:20PM (#50064919)
I haven’t had this happen yet, but I was just thinking about this last night. I would be very interested to see if exercising TDD, as difficult as that practice is to actually do, would have a meaningful positive impact on improving this phenomenon. Usually agents model what most humans do, writing tests to pass and assert that the code you just wrote works as you wrote it.
Re:AI coding agents generate more code, but not more software by jacquesm on Monday October 12, 2026 @10:10PM (#50065730)
It does. But keep an eye on whether the tests actually run.
Re:AI coding agents generate more code, but not more software by bunderbunder on Monday October 12, 2026 @08:47PM (#50065124)
I’ve had decent luck with it as long as I at least stub out the test cases with a test name and a “given … when … then” comment.

Otherwise it seems the AI has a tendency to write the worst kind of TDD antipattern microtests that simultaneously make refactoring impossible while failing to document or even confirm the most important functional requirements.

Once it’s got those stubs it does do a pretty good job of suggesting behaviors I forgot to specify, and other improvements. It’s just “designing” test fixtures from the ground up that seems to put it into “generate plausible-looking slop” mode.

Re:AI coding agents generate more code, but not more software by joshAg on Tuesday October 13, 2026 @11:11PM (#50081937)
It has the same failure mode as dealing with TDD by a human engineer, which includes things like mocking the function under test in the test and then testing the mock, asserting only things that aren't actually exercised by the function under test (like asserting the initial user ID is the same as the final user ID for a function that changes name of a group), not validating side effect behavior (or lack thereof) for a function that's only purpose is the side effect (like appending a log line to a log file), and more!

What changes with TDD is that the important part of the review moves from the mainline code to all the TDD tests and the coverage, and you also need to see a test run from before the code changes with all the new/updated TDD tests failing in addition to just a test run after the changes with all the (at least) new/updated TDD tests passing.

Re:AI coding agents generate more code, but not more software by bunderbunder on Wednesday October 14, 2026 @10:25AM (#50088250)
This is part of the reason that I've come to view mocks as a code smell. They're too easy to screw up.

But even more than that, I've found that modules that need to be tested with mocking can often be reorganized to be testable without mocking. And doing so tends to also make the code easier to understand and easier to modify without breaking things in the future.

Re:AI coding agents generate more code, but not more software by joshAg on Wednesday October 14, 2026 @01:48PM (#50091829)
Mocks/stubs definitely have a smell to them, but especially if you want to do more than just unit tests can be necessary. I've found that setting up the mainline architecture so that components that are allowed to be mocked have a single access path in the code makes it easy to mock at a level low enough that the only thing being mocked is that component and then preemptively building out those mocks into the test framework works decently well. For me, 3rd party services are usually fine to mock, but things like DBs and redis are usually not. I'd rather spend the time to make a test container the devs can run locally so that they can't skip over checks/constraints and the integrations with services/components we have full control over.

That way you can tell at a glance of the test file change just how bad it is. Seeing new raw mocks (eg sinon in the test file at all) instead of the existing mocks is an instant no, and seeing an existing mock getting added requires at least investigation for necessity.

Re:AI coding agents generate more code, but not more software by eivindmeyer on Wednesday October 14, 2026 @06:41AM (#50085453)
Classic assert true theatre lol