News for hackers. Stuff that matters.
Re:Why Most Unit Testing Is Waste [pdf] by AnimalMuppet on Tuesday October 31, 2017 @03:51PM (#15595995)
> Here's the crux of your argument - and I honestly think it's a flawed premise: The failure of unit tests indicates something other than "Something Changed".

No, the failing tests indicate something changed, and where it changed - what external behavior of a function or class changed. Is the change the right thing, or a bug? Don't know, but it tells you where to look. That's miles better than "I hope this change doesn't break anything".

> "design your APIs before implementing them" - this only works out if we know our requirements ahead of time.

No. "API" here includes things as small as the public interface to a class, even if the class is never used by anything other than other classes in the module. You have some idea of the requirements for the class at the time when you're writing the class; otherwise, you have no idea what to write! But writing the test makes you think like a user of that class, not like the author of that class. That gives you a chance to see places where the public interface is awkward - places that you wouldn't see as the class author.

> This tells me that we're rewarding the wrong thing: the creation of passing tests, not the creation of correct code.

We're rewarding the creation of provably working code. I fail to see how that's "the wrong thing".

Re:Why Most Unit Testing Is Waste [pdf] by falcolas on Tuesday October 31, 2017 @04:33PM (#15596300)
> where it changed

If we were discussing integration tests, where the interactions between different methods and modules are validated against the input - I'd agree.

But this is about unit tests, in which case the "where" is limited to the method you're modifying, since it's most likely mocked out in other methods and modules to avoid tight coupling.

> includes things as small as the public interface to a class

If we're talking about internal APIs as well, we can't forget that canonical unit testing and TDD frequently requires monkey patching and dependency injection, which can make for some really nasty internal APIs.

> I fail to see how [provably working code is] "the wrong thing".

So, thinking back a bit, I can recall someone showing me TDD, and they gave me the classical example of "how to TDD": Write your first test - that you can call the function. Now test that it returns a number (we were using Python). Now test that it accepts two numbers in. Now test that the output is the sum of the inputs. Congratulations, you're done!

Except you're not, not really. What happens when maxint is one of your parameters? minint? 0? 1? -1? maxuint? A float? A float near to, but not quite 0? infinity? -infinity?

Provably working code is meaningless. Code that can be proven to meet the functionality required of it - that's what you really want. But that's hard to encapsulate in a slogan (like "Red, Green, Refactor"), and harder use as a source of quick dopamine hits.

Re:Why Most Unit Testing Is Waste [pdf] by AnimalMuppet on Tuesday October 31, 2017 @04:57PM (#15596492)
> But this is about unit tests, in which case the "where" is limited to the method you're modifying...

Sure, but the consequences may not be. Here's a class where you have two public functions, A and B. Both call an internal function, C. You're trying to change the behavior of A because of a bug, or a requirement change, or whatever. In the process, you have to change C. The unit tests show that B is now broken. Sure, it's the change to C that is at fault, but without the tests, it's easy to not think of checking B. That's the "where" that the tests point you to.

> Provably working code is meaningless. Code that can be proven to meet the functionality required of it - that's what you really want.

Um, yeah, of course that's what you really want. Maybe you should write your tests for that, and not for stuff you don't want? If you don't care about maxint (and you're pretty sure you're never going to), don't write a test for maxint. But it might be worth taking a minute to think about whether you actually do need to handle it (and therefore test for it).

Re:Why Most Unit Testing Is Waste [pdf] by ekidd on Wednesday November 01, 2017 @07:49AM (#15600347)
> canonical unit testing and TDD frequently requires monkey patching and dependency injection

This is probably the root of most of our disagreement. I belong to the school of thought that says (in most cases) "Mocking is a Code Smell": https://medium.com/javascript-scene/mocking-is-a-code-smell-... And dependency injection (especially at the unit test level) is giant warning sign that you need to reconsider your entire architecture.

Nearly all unit tests should look like one of:

- "I call this function with these arguments, and I get this result."

- "I construct a nice a little object in isolation, mess with it briefly, and here's what I expect to happen."

At least 50% of the junior developers I've mentored can learn to do this tastefully and productively.

But if you need to install 15 monkey patches and fire up a monster dependency injection framework, something has gone very wrong.

But this school of thought also implies that most unit tests have something in common with "integration" tests—they test a function or a class from the "outside," but that function or class may (as an implementation detail) call other functions or classes. As long as it's not part of the public API, it doesn't need to be mocked. And anything which does need to be mocked should be kept away from the core code, which should be relatively "pure" in a functional sense.

This is more of an old-school approach. I learned my TDD back in the days of Kent Beck and "eXtreme Programming Explained", not from some agile consultant.

Re:Why Most Unit Testing Is Waste [pdf] by MaulingMonkey on Wednesday November 01, 2017 @01:21AM (#15598954)
> If we were discussing integration tests

I feel like you're perhaps defining unit tests to exclude "useful unit tests" by relabeling them. Yes, if you exclude all the useful tests from unit testing, unit testing is useless. Does a unit test suddenly become a regression test if the method it tests contains, say, an unmocked sprintf - which has subtle variations in behavior depending on which standard library you link against? No true ~~scotsman~~ unit test would have conditions that would only be likely to fail in the case of an actual bug?

> But this is about unit tests, in which case the "where" is limited to the method you're modifying

That's still useful. C++ build cycles mean I could have easily touched 20 methods before I have an executable that lets me run my unit tests, telling me which 3 of the 20 I fucked up in is useful. Renaming a single member variable could easily affect that many methods, and refactoring tools are often not perfectly reliable.

Speaking of refactoring, I'm doing pure refactoring decently often - I might try to simplify a method for readability before I make behavior changes to it. Any changes to behavior in this context are unintentional - and if they occur, 9 times out of 10 discover I have a legitimate bug. Even pretty terrible unit tests, written by a monkey just looking to increase code coverage and score that dopamine hit can help here - to say nothing of unit tests that rise to the level of being mediocre.

Further, "where" is not limited to "the method you're modifying". I'm often doing multiplatform work - understanding that "where" is in method M... on platform X in build configuration Y is extremely useful. Even garbage unit tests with no sense of "correctness" to them are again useful here - they least tell me I've got an (almost certainly undesirable) inconsistency in my platform abstractions, likely to lead to platform specific bugs (because reliant code is often initially tested on only one of those platforms for expediency). This lets me eliminate those inconsistencies at my earliest convenience. When writing the code to abstract away platform specific details, these inconsistencies are quite common.