News for hackers. Stuff that matters.
Re:My AI skeptic friends are all nuts by intrasight on Monday June 02, 2025 @11:59PM (#44166128)
My philosophy is to let the LLM either write the logic or write the tests - but not both. If you write the tests and it writes the logic and it passes all of your tests, then the LLM did its job. If there are bugs, there were bugs in your tests.
Re:My AI skeptic friends are all nuts by dfedbeef on Tuesday June 03, 2025 @12:18AM (#44166217)
This is pretty nifty, going to try this out!
Re:My AI skeptic friends are all nuts by underdeserver on Tuesday June 03, 2025 @12:50AM (#44166409)
That rather depends on the type of bug and what kinds of tests you would write.

LLMs are way faster than me at writing tests. Just prompt for the kind of test you want.

Re:My AI skeptic friends are all nuts by catlifeonmars on Tuesday June 03, 2025 @04:07AM (#44167599)
Idk about you but I spend much more time thinking about what ways the code is likely to break and deciding what to test. Actually writing tests is usually straightforward and fast with any sane architecture with good separation of concerns.

I can and do use AI to help with test coverage but coverage is pointless if you don’t catch the interesting edge cases.

Re:My AI skeptic friends are all nuts by therealmarv on Tuesday June 03, 2025 @03:24AM (#44167330)
I don't agree. What I do agree on is to do it only with one LLM.

Quality increases if I double check code with a second LLM (especially o4 mini is great for that)

Or double check tests the same way.

Maybe even write tests and code with different LLMs if that is your worry.

Re:My AI skeptic friends are all nuts by joshribakoff on Tuesday June 03, 2025 @12:28PM (#44171797)
Disagree. You could write millions of tests for a function that simply sums two numbers, and it’s trivial to insert bugs while passing that test.
Re:My AI skeptic friends are all nuts by lelanthran on Wednesday June 04, 2025 @08:09AM (#44179839)
> My philosophy is to let the LLM either write the logic or write the tests - but not both. If you write the tests and it writes the logic and it passes all of your tests, then the LLM did its job. If there are bugs, there were bugs in your tests.

Maybe use one LLMs to write the code and a wildly different one to write the tests and yet another wildly different one to generate an English description of each test while doing critical review.