What does that mean, exactly? Any code you write is obviously in the service of some goal. Some code, like a method to split a String that's part of a standard library, is its own end: it's a complete API, so it should be tested as such.
A whole lot of other code, though, is in service of some larger component. Maybe you've got 15 classes that all work together to form the e-mail component, and hopefully only one or two of those classes really encapsulate the external API (whether it's an explicit API or just implied) exposed to other components, and the rest are logically part of the implementation. So instead of unit testing every one of those, you get more mileage testing at the level of that API. That lets you change the implementation of that component without having to rewrite a bunch of tests, and the tests themselves then become a much clearer documentation of that API.
But plenty of times, the overall API is so complex that there's a combinatorial explosion if you try to test at that higher level; in that case you have to find smaller chunks that have few enough paths through them that they can be tested thoroughly. Sometimes those chunks are the level of a single class or method; often they're still a collection of things.
Rather than starting from the classes and say "let's test everything," you look for logical groupings that form logical or explicit APIs, say "let's test this API," and then push the tests down a level only insofar as you can't test that API thoroughly enough to execute all the code paths in the related code.
To put that yet another way: test APIs, not implementation details, and the more things you can treat as implementation details, the less friction your tests will exert on your development process and the more value you'll get from them as a result.
I think the cost of this is far higher than it's worth.
But note that once you you have an organization good at writing tests, the incremental cost of writing a test is very, very small.
We like the term "10x" around here. Design systems for 10x the customers of today. Write code that will survive for a year. (Otherwise it may have been wasted.) Only fix bugs once so you can grow the engineering team without rehashing old issues.
Tell me what software you write so that I know to never, ever use it.
I question the design decisions of any code base with tens of thousands lines of code.
What operating system are you running on?
I'm also curious how modular it is. Does a kernel hacker have to keep the structure of that whole set of code in his/her head? I'm guessing there are a lot of clean seams in it that in practice reduce the actual lok related to a project down to a much smaller number.
"With the 2.0 kernel linux really grew up a lot. This was the point we added loadable kernel modules. This obviously improved modularity by making an explicit structure for writing modules. Programmers could work on different modules without risk of interference."
When someone pointed out that linux kernel had millions of lines now you are retreating to "but it is modular". Programming languages have had modules since the eighties. If someone points out a 25,000 line module what will you say? Each function has less than 10 k lines?
The point is that there are well designed code bases (the design includes modularization) with sizes well above 10 k lines , which directly contradicts your point. I've worked on well designed enterprise systems with a few million lines of code.
I won't debate a strawman anymore so I'll move on. Btw, are device drivers counted in that millions of lines of code? If so those are very independent things.
This thread was about test coverage, how many of those millions of lines of kernel code are covered by a unit test?
EDIT: so you agree with my original assertions that large test suites may not be worth the cost?
Not many. And yet the kernel is an awesome piece of code. That was the point. I don't have the time to debate test-driven/test-coverage fanatics either.
(edit: twice as slow is about the same as twice as expensive)
Shipping bugs is not a show stopper. It's not even serious. What is serious is developing a live, production app without any consideration for bugs: backups (at least a couple of very different kinds) and control keys. A control key can be anything which will be visibly not ok when there's a bug somewhere.
http://www.startuplessonslearned.com/2008/11/five-whys.html
Every time you ship a bug to customers, ask yourself "How will we prevent this bug from ever reaching customers again?" and "Why did we write this bug in the first place?"
Almost every time, unit or acceptance tests would have caught the bug initially or prevented the regression. That's not to say unit tests are the only solution.
Sometimes:
* you need better knowledge sharing between engineers
* you should spend more time refactoring your code
* your internal APIs are confusing and inherently cause bugs
Often a follow-up to a bug will involve changes to all of the above.
When you start an experimental project, don't focus on automated tests if you don't need to... I'm all for lazily pulling in the processes you need as you need them. :)
Whether you write unit tests or not is entirely context-sensitive. The author's points might make sense in some context, but they certainly don't apply to every team or even every engineer.
Maybe this guideline is helpful: "I just wrote a new component. What kind of test coverage do I need to allow a new engineer to refactor arbitrarily and still be convinced my code works?"