News for hackers. Stuff that matters.
Re:GIMP now has UI tests by samuria on Friday November 20, 2026 @12:09AM (#50570411)
It’s frustrating to create tests with models, not sure if it’s everyone’s experience, but every time I asked it to create tests, it makes the tests conform to the sometimes broken code. Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.
Re:GIMP now has UI tests by Waterluvian on Friday November 20, 2026 @12:12AM (#50570434)
It can’t magically know expectations. If the code is well documented or there’s decent associated documentation, I’ve found it has no issue writing tests.
Re:GIMP now has UI tests by t43562 on Friday November 20, 2026 @03:42AM (#50571879)
Tests are usually the best documentation and often the only documentation. If humans bother to write anything it should be the tests, not ambiguous human language...beyond the high level 1-liner overviews.
Re:GIMP now has UI tests by alyandon on Friday November 20, 2026 @11:24AM (#50577157)
I'm actually surprised at how well non-frontier/local models are able to infer the intent of code from the structure, method names and sparse random comments to the point that they are capable of writing reasonable tests.
Re:GIMP now has UI tests by forgotmypw17 on Friday November 20, 2026 @12:14AM (#50570444)
I recommend iterating on your prompts.

Prod your agent first until it can explain back to you what's wrong with the tests.

Then ask your agent to write a better prompt for writing tests (and save it to an .md file)

Then reference that .md file and iterate on the tests again. It may take a couple of tries, but I've had good results with this strategy.

Ended up with a whole prompting framework.

Re:GIMP now has UI tests by skydhash on Friday November 20, 2026 @12:46AM (#50570631)
Prodding an agent vs doing interesting things with my time? What should I choose?

To be fair, some people like to prod. But, for me, I'd rather take hours reading manuals than do trial and error prodding for 30 minutes (even if I knew it would be that short). Yes it's not optimal in terms of time, but it's better for my soul and well being.

Re:GIMP now has UI tests by 0xcoffee on Friday November 20, 2026 @12:55AM (#50570681)
[delayed]
Re:GIMP now has UI tests by thisismyanon2 on Friday November 20, 2026 @03:01AM (#50571540)
Perhaps because they're more interested in learning something themselves.
Re:GIMP now has UI tests by HiPhish on Friday November 20, 2026 @01:35AM (#50570933)
> Yes it's not optimal in terms of time, but it's better for my soul and well being.

It is also optimal in terms of time in the long run. The prodder will have to prod over and over again because he has learned nothing. You on the other hand will actually understand what is going on and will be able to either do it yourself or give precise instructions to the machine next time. Don't let yourself become a meat proxy.

Re:GIMP now has UI tests by doginasuit on Friday November 20, 2026 @01:16AM (#50570820)
The problem I've had is even when it can explain back the issue, it still writes for itself awful instructions with extra fluff, strange points of focus, and borderline misinterpretations of what I said.

Maybe that's not a big deal, they can still be generally useful, but they seem important to commit with the project if they will be used going forward. I hate the idea of a massive amount of slop riding along with the code, and slop begets slop.

The idea I'm chasing now is writing the documents myself with the hope that enough examples and a document about writing documents will be enough to allow it to write clearly.

Re:GIMP now has UI tests by hulitu on Friday November 20, 2026 @12:56AM (#50570686)
> Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.

It is the same like humans: if they say your code is wrong, you will hire someone else who doesn't find so many bugs. It doesn't look good.

Re:GIMP now has UI tests by alerighi on Friday November 20, 2026 @03:29AM (#50571773)
True. It's tough useful to build non-regression tests, that is build tests that test the current behavior, assuming it correct, to possibly spot bugs in the future.