And clang is really great at catching more issues than other compilers would, and then reporting them better than the others. So the title isn’t far off.
I don't strictly disagree with your first point, but I disagree with the attitude from many dev regarding tests that lead to it becoming too common like in this article (and I'm not piling on the guy, it's very very common).
It's like people have forgotten what tests are for and thus what you want from them: a test that pass is essentially useless, it completes the green check mark and that's it, it doesn't matter. It's the test that fails that provide you information, that matters to you.
One might be tempted to disagree, tell me that passing test also provide information, by ensuring this and that and I will insist that they take a second and realize that no, they don't. Ultimately, the only value they have is in fact that they can fail, it's always the failure that matter.
There are indeed tests which fails to fail for other unplanned reasons, by that's not really what I'm getting into, I'm aiming specifically for the attitude of tests for the sake of tests.
> It's like people have forgotten what tests are for and thus what you want from them: a test that pass is essentially useless, it completes the green check mark and that's it, it doesn't matter
This is, in my opinion, not true in some cases. Sometimes, I'll write unit tests that never fail, and are purely to allow me to verify one of my assumptions about the code. Basically, the unit test is a tool to force me to think through each line of the code, and that's it.
Typically, when I do the above, I also use a debugger and/or line coverage tool to ensure the test actually reaches part of the code I'm trying to reason about, and with the debugger, this sort of "test-writing as a tool to think about code" is in my opinion much more powerful than just reading code. I do this a lot in code review when I don't understand someone's code, since it's easier to run sample values through it and set breakpoints in a test, than it is to do all that in my head looking at the diff view.
Now, you might "no true scottsman" argue that those are not tests (and indeed, I often don't bother to check them in), but I think they are.
I also think integration and blackbox tests that never fail are fine, and expected. If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails?
Ideally yes, to test the uptime monitor!
That is fine and has value, but that is usually called an assertion (and most language have them under some sort of assert() function), which is different from an unit test.
> If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails?
How do you know if it works otherwise ? In fact for that specific exemple, the most naive version of that test actually misses many failure cases as we've seen again and again in the last decade with status page that fail to detect any error
unit tests let me pick sample values, and I do what I'm talking about using unit test libraries, not assertion libraries. Perhaps you misunderstood me.
> many failure cases as we've seen again and again in the last decade with status page that fail to detect any error
That is a political problem, not a testing problem. The majority of status pages that fail to detect errors are because the company decided to have a human update it because the test failed too often, or did not differentiate "error" and "warn" or provide a human description.
Status page uptime is often a political matter for a company, and no unit testing practices will fix a political problem.
But there are many subtle reasons why a test might not get written. Sometimes it’s deadlines. Sometimes it’s because a component is hard to test in isolation. Sometimes it’s just an honest mistake.
So, while I would encourage everyone to write tests I also won’t judge them for not doing it.
But it's also important to remember why you write test: for their failure cases, not their success.
This code will not pass a code review in my last shop:
1. First get the return code: rc = memcmp(…
2. Then compare, and please put the constant first: if( 0!=rc ) …
C and C++ are dangerous tools, you should not use in production all the “sugar” they provide.
> And this really ticks me off for some reason, probably because the entire web and beyond has moved to that form of expression where one exaggerate and then say "yeah but I have my reason". I miss adult discourse.
I completely agree. I've seen a lot of people try to excuse being rude to people because they're extra emotional at the moment. If you're extra emotoinal at the moment, *don't write it*. It's more reasonable to use as a post-facto reason. (Example: Sorry. Yesterday I wasn't in my right mind and I said some mean things to you that I regret saying. It won't happen again.) But it's not at all okay to try to excuse what you're about to write. That's not an apology at all. That's like a person with anger management problems trying to excuse being able to hit people whenever they want.
This was always adult discourse.
We just have rose colored glasses to think otherwise.
They did not say they didn't do that, you're assuming.
If I were the person here, I might write a simple test with 'val1="foo"; val2="bar";' to make it fail. That would make it fail because the first byte is different, while I would think it failed because the entire value is different.
You might say that I should have also tried "bar" and "baz" (which would reveal the issue), but it's infeasible to try all possible inputs.
Most code is not written with exactly an excess of time to follow perfect practices every time, and so we must make tradeoffs.
In the article, the author confirms his test was not working:
> But that bug in my code ruined one month of my work. I coded without realizing tests don't work at all. OUCH!
The issue is exactly what parent mentions, that OP did not let his test fail first so that they were never confident in that the test worked.
This is absolutely the wrong way to do testing and the crux of what I was getting at: you should not change a single line of code within the test itself when making them pass or fail.
If you change the test itself, then it's not the same test, you have not ensured your test works, only that another test works.
I'm sorry if this sounds harsh or if my phrasing is not correct, but it's important to remember this. If "testing your test" implies "making modification to the test", then you're not doing it right.
Using Clang instead of GCC won't make him a better engineer.
> And this really ticks me off for some reason, probably because the entire web and beyond has moved to that form of expression where one exaggerate and then say "yeah but I have my reason". I miss adult discourse.
I'm on the same boat with you. Worse, people writing these titles also hold to their provocative titles firmly, making discourse almost impossible. Trying to read this loaded articles from title to last dot is exhausting and counterproductive, too.
This is why I also don't publish anything unless I can write a neutral title, and distill the post's tone to a neutral point.
> But that bug in my code ruined one month of my work. I coded without realizing tests don't work at all. OUCH!
Sorry to say but that too is your fault. First thing you're supposed to do with tests is make them fail, then make them pass. The only safe way to handle it is that if you've never seen your test as written fail, then it never fails.
As an aside, the first line says:
> Sorry for the provocative title, but I'm too emotional these days.
And this really ticks me off for some reason, probably because the entire web and beyond has moved to that form of expression where one exaggerate and then say "yeah but I have my reason". I miss adult discourse.