News for hackers. Stuff that matters.
Re:Questions to ask yourself when writing tests by arkh on Tuesday January 02, 2018 @04:01AM (#16050782)
> Because most of the time, it reduces "test precision". Which means when such a test detects an issue, there's now a bigger area of the code where the issue could be located.

Tests are not a debug tool. Tests are here to tell you when you broke something.

When this happen you can get your debugging toolbox out: stack traces, profilers and things like GDB. And then follow the steps your test script did.

Re:Questions to ask yourself when writing tests by amarkov on Tuesday January 02, 2018 @06:35AM (#16051294)
All the projects I've most enjoyed working on had clear, explicit test failures, which almost always gave me enough information to hunt down what I'd broken and where. Obviously that's not realistic for some kinds of projects. But even in a legacy C project, if I had to pull out stack traces and profilers and GDB every single time I wanted to understand why a test failed, I would say I'm working in a pretty terrible codebase.
Re:Questions to ask yourself when writing tests by newfoundglory on Tuesday January 02, 2018 @04:28PM (#16055874)
I disagree and am horrified by the dystopic image of our entire test suite having two possible results: "everything passed" or "something is wrong".
Re:Questions to ask yourself when writing tests by arkh on Tuesday January 02, 2018 @06:03PM (#16056716)
Not two results.

But "when you click on this after that this happen while it shouldn't" is usually enough info to start debugging: you have reproducible steps. Which you can do with a debugger running so you see exactly were and how things break.

And it is a lot less brittle than "well we tried to refactor some minor thing and now everything is red; but we don't know if the software behavior changed or just because our tests were just checking the implementation".

Re:Questions to ask yourself when writing tests by newfoundglory on Wednesday January 03, 2018 @03:52AM (#16059500)
well, if you're determined to write stupid tests you can do it at any level. And that is not enough info to start debugging unless you had the machine in a known and reproducible state to begin with, which is where we get back to writing focused reproducible tests.