Scrapping unit tests is scrapping time.
The correct solution to this is to not throw passing tests out, but to stop doing white-box testing.
(Not to mention that a test that may pass in your CI environment may fail - frequently - in your local workspace.)
None. But that's hardly the issue, I too often have seen people championing heavy testing as a way to deliver quality software, because it's the agile way. The problem is not with the philosophies but the way people interpret them or misinterpret them.
The sarcastic quote in the article sums my feeling on the behaviour I've seen in a lot of shops: "I find that weeks of coding and testing can save me hours of planning.".
I've had the hardest time convincing people to spend a week thinking about our approach and solution, but adding months of development time to expend test coverage and suddenly everyone thinks it's worth it (And usually with nothing to back that up).
A lot of us would consider SQLite to be high quality and relatively bug-free. The extensive test suite they've built up that exercises each release is a huge reason for it.
Of test code quantity, Coplien writes:
>If your coders have more lines of unit tests than of code, it probably means one of several things. They may be paranoid about correctness; paranoia drives out the clear thinking and innovation that bode for high quality.
> - Keep regression tests around for up to a year
> - Throw away tests that haven’t failed in a year.
SQLite appears to keep their tests. Somebody files a bug; SQLite writes a test that reproduces that bug; the test code remains long after the bug is fixed. This prevents the bug from reappearing. (Isn't that the main purpose of "regression" in the phrase "regression test"?)
I also think it's better to keep relevant regression tests for years. E.g. Consider that there's piece of code that's been working correctly for 5 years with 5 years of passing regression tests . Imagine a new programmer wants to rewrite the code to optimize it for speed and reduced memory usage. I think we'd feel much more confidient if the new code passes those same regression tests that were accumulated over 5 years.
As for test code size ratio... A lot of good comprehensive tests will have LOC outnumbering the actual code being tested. This is especially true for library code that's used in many places up the stack. I wrote string parsing routines and a reverse Boyer-Moore search routine where the test code (test edge cases, test nulls, test string sizes at 2^32 boundaries, etc) was 10 times larger than the actual code.
Of testing's utility, Coplien writes:
... which looks like a strawman and a false dichotomy. Can anyone cite a credible development philosophy that believes testing can replace bad developers or bad process?We could say that about <ANYTECHNOLOGY> such that <ANYTECHNOLOGY> can't replace quality developers. Garbage Collection doesn't improve quality, developers do. Array boundary checking doesn't improve quality, developers do. And so on.
Or maybe there's a difference in terminology? I wonder if Coplien considers SQLite testing "system test" or a "unit test"? Does he consider SQLite "white box testing" or "black box testing"?