Re:GIMP now has UI tests
by locusofself on Friday November 20, 2026 @12:27AM
(#50570515)
Not using AI to at least writes tests in this day and age is crazy work. I started a desktop application 2 months ago and it has almost 3,000 tests now, MANY of them UI tests.
[ Parent
| on HN ]
Re:GIMP now has UI tests
by timcambrant on Friday November 20, 2026 @12:41AM
(#50570588)
AI is a bit too excellent at writing tests. The number of tests doesn't really convey any information on test coverage. At least I've found that agents that naively improve codebases will write tests for pretty much anything, including irrelevant behavior that has little value in ensuring stability, security or correct behavior. It would probably help to run a test review stage with a strong model, where agents attempt to correct and consolidate tests to guard behavior rather than ensuring that theoretical code paths are followed.
In my experience, designing tests cases to ensure that they're othogonal and verify your invariants (domain or technical) is very mentally taxing and not something you can truly delegate. The actual typing maybe, but testing is a lot of duplicate code and you can go far with just copy-pasting stuff around and building helpers.
Re:GIMP now has UI tests
by aidenialislife on Friday November 20, 2026 @12:56AM
(#50570687)
Designing for testing helps as well. Pure functions for core logic, coding against interfaces etc.
I my opinion, AI only changes the ease of writing these tests, not really the quality.
I find that the human written tests on my project were the same. We were guarding not just the business behavior but also weird cases that have nothing to do with business, and also self-evident cases that basically boil down to x=x or assert(true).
Not to mention the stupid side-effect of not updating mocks when the APIs move but tests don't test that part so they are okay for a lot of time, and suddenly after a non-related change they are not.
Re:GIMP now has UI tests
by locusofself on Friday November 20, 2026 @01:02PM
(#50579090)
I agree, some of my small projects have had an explosion of tests. It usually doesn't cause my any problems other than waiting for tests to run, but that isn't great.
Look, guys, you are literally allowed to fork and go ham on the code.
Re:GIMP now has UI tests
by mepiethree on Friday November 20, 2026 @01:30AM
(#50570901)
Yeah if people can make a better and more popular Gimp in an afternoon, go do it.
Gimp has millions of downloads this month. They are doing something right.
Using AI to write tests in this day and age is crazy work. The machine is great at writing tests, but the quality of the tests is awful. It will write redundant tests, it will write tests that assert too many thing in one, it will not write tests that need to be tested. It's the part where I have to clean up the most every time. Quantity is not quality, and you really don't want to skip on the quality of your quality assurance. I'm writing a React + Django full-stack application, so there should be more than enough training data out there.
Re:GIMP now has UI tests
by byronturncoat on Friday November 20, 2026 @09:08AM
(#50574859)
Some people refuse to use AI for ethical, environmental, or social reasons. It isn't crazy to want your projects to be built by humans