OK, sure. I'm basically pro-TDD, seeing as how everyone's taking sides. Yet there's a problem here, which is that (as Martin says here) bad testing is basically a design problem. To write better tests, learn better software design - which is to say, you do not do it (or at least most efficiently) by doing more TDD.
This stuff is not in the brochure. And it should be; because without it, the claims made by TDD evangelists are somewhat misleading. No, it will not make you a better engineer on its own, at all. No, it is not a substitute for getting your whiteboard marker out and drawing a few boxes, or thorough code reviews, or whatever else.
Sandi Metz makes this point in Practical Object Oriented Design in Ruby[0]; and the chapter on testing is right at the end of that book, presumably for the same reasons (get the design sense first, then you'll get a sane red-green-refactor thing going).
Note that I'm not talking about Ian Sommerville here - I'd never heard of him until this whole contretemps, but I'm given to understand he knows a thing or two about design. Perhaps if he really stuck at TDD, Bob Martin's favourite bit would flip[1]. For less experienced programmers, who have not yet felt the pain of maintaining a ball-of-mud and learned a thing or two about separating concerns, TDD is not going to help without ongoing education about design.
[0] http://www.poodr.com/ . The design principles are different outside of the OO world, of course; but not the principle of design!
[1] http://blog.8thlight.com/uncle-bob/2012/01/11/Flipping-the-B...
It forces a very short feedback loop between acts of design and experiencing the design as a consumer. That gives people immediate feedback on bad design choices, giving them opportunities to see the problems they're creating.
It also forces people to think immediately about the consumer perspective of a design rather than the implementation perspective. Instead of mentally being inside the objects and methods they're building, they have to start thinking about them from the outside.
I agree it's not a substitute for getting up and going to a whiteboard. But, then, going to a whiteboard is not a substitute for TDD. They're mainly working at different levels of design; when I'm at a whiteboard we're generally talking about the relationship of relatively large pieces of a system. Whereas during TDD you're mainly confronting the fine details of design.
I guess I loosely fall into the category of "anti-TDD". I don't generally go around badmouthing it, but I'm not really sold on the idea that it would save me significant time or effort. I certainly encounter things that TDD would likely catch earlier, but those are typically easy fixes and caught in QA or beta testing. The majority of my bugfixing hours are eaten up by things for which I don't even know what it would mean to write tests, such as:
- reproducing rare hardware-involved errors (often race conditions, often based on the variability of random-ish real-world events) in a way that allows getting relevant information logged/dumped
- clarifying product definition/spec, usually to resolve some inconsistency where the behavior isn't invalid per se but implicitly conflicts with some other requirement (i.e. we basically decide which behavior to call a bug and change that one)
- reverse-engineering undocumented design decisions of some third-party product that our product is expected to work with, because we foolishly believed that there was a standard (overlaps somewhat with #1)
So while I don't claim that TDD has no value, I admit to rolling my eyes a bit at the more fervent evangelists.
We have over 30,000 tests and we've had to rewrite part of it because it memory-leaked so badly. Unfortunately the company has strict rules against contributing back to Open Source projects.
Specifically the Uncle Bob example of OOP run amok.
Free but very buggy.
I'm not an anti-TDD zealot, but I don't personally do it, nor require any of my developers to do it (but they can, if it helps them).
As with everything in software, some things work well for some people, and not so much for others.
Instead of TDD, HDD works best for me. https://www.youtube.com/watch?v=f84n5oFoZBc
http://patrick.lioi.net/2011/11/23/guard-rail-programming/
TDD has always seemed to me to be another sketchy, cult-like following in the Agile ecosystem. I've never seen anybody be productive at it, even if they did find some kind of solution to a problem long after they would have doing normal development.
But what do I know, I'm not even a fan of unit tests. I'd rather go to the real-deal and run some acceptance/integration test at 2 in the morning.
Another thing I don't like about tests are categories of tests that should be automated by the compiler, tracer or similar.
Like in python you type check a lot in unit tests, but in a statically typed language, the compiler does your type tests for you already so you don't have to write that kind of test any more.
There are also tests I call 'breakpoint equivalent testing' where you put expectations of certain methods getting called in a method. I wish I could just set some breakpoints in an IDE and have some recorder automatically write the method call expectation code for me.
Writing software is not like driving a car. It is like engineering a car. Making a change on the assumption that tests will tell you if there is a problem is like deciding to move the gas tank and saying, "If that turns out to be an issue, QA will tell us when they do their collision tests." (Sort of like that. Obviously less expensive.) When you are making a change, you need to think about whether that change will require any new tests and you need to be thinking about how it integrates into the system as a whole. Maybe moving the gas tank means a new kind of collision test needs to be added to the repertoire.
Because the metaphor was bad, the GP seemed to be taking it as a criticism of testing, which isn't reasonable. But I may have been mistaken about his intent.
On the other hand, if the change you are making is just altering the shape of the bumper for cosmetic reasons, it seems pretty reasonable to assume that QA will tell you if it adversely impacts safety in ways that were not obvious.
I agree about static types. Static types are basically a way of having the compiler automatically write and run whole classes of unit tests.
I think where tests really help is when it's not possible to sensibly reason about the system because [looks at current codebase] e.g. it's horrifying interwoven ball of mud that's never heard the word "no" or met a Perl module or methodology it didn't want to halfheartedly adopt in the last 10+ years.
The act of TDD, or test compliance is only that you've looked at everything twice. It doesn't guarantee quality. Writing modular code means less friction over time, and TDD encourages this, but it's not necessarily a requirement imho.
Do you actually apply that reasoning in everything you do? This is highly unproductive approach.
At least one paragraph from [1] this essay by PG is very relevant:
Most people have learned to do a similar sort of filtering on new things they hear about. They don't even start paying attention until they've heard about something ten times. They're perfectly justified: the majority of hot new whatevers do turn out to be a waste of time, and eventually go away. By delaying learning VRML, I avoided having to learn it at all.
Great quote, but VRML impacts the actual work of the programmer in his career about 1%. TDD on the other hand can not only impact your work but take you to another level in nearly every task.
I haven't spent time learning Rust or Go yet. My clojure and haskell skills are subpar. I can't know everything. But TDD is a practice that applies regardless of language or framework. Languages come and go, but practices are what help you really hone your craft.
I do it, and I find it valuable. But I think if you're not using it as a design tool, you're missing most of the benefit.
You don't do TDD (IMO) to prove correctness. You do it to achieve a level of composition and design that makes later requirements changes and refactoring less painful. Proving that your stuff actually works is a convenient side effect. But not the primary goal because tests have costs.
I haven't followed Osherove for years, so maybe his thinking has changed, but it used to be common to think of chasing high test coverage numbers as an anti-pattern. 70 to 80% is the sweet spot. You're designing. You're being productive. That last 10, 20 or 30% of coverage gets exponentially more expensive both to develop and maintain, and it provides no additional design benefit. It's only testing for it's own sake.
BDD and the culture that sprang up around it (at least in the Ruby community) was such a disappointment after having done TDD for a few years previous. It's like every known anti-pattern was adopted as a core deliverable and the entire point of the exercise (IMO, solving the "blank page" problem in building systems; Design) was forgotten entirely.
It's my experience that Developers who've adopted this style of "TDD" are cargo culting, and incredibly difficult to work with. Reason didn't get them into the belief. They can't be reasoned out of it. It's religion by that point.
Now that I'm in Scala, and TDD my own code, I'm a very happy developer. I never suffer blank-page issues. I never bother writing tests after the fact. I don't often encounter bugs in such code, and they're almost never fundamental design issues when I do. And my coverage proudly hovers around 70%.
That is how you TDD right IMO. Follow the lessons learned a decade or more ago and avoid the anti-patterns.
Chasing test coverage is not only a good way to light piles of somebody else's money on fire by wasting time for little benefit, but it's actively harmful to the quality of your code base over the long term. You're disincentivized to correct design mistakes, and you're encouraged to over design to enable a level of testing granularity that never paid any concrete benefits for itself in the first place.
I'm a solid proponent of TDD then. But it's like saying I think water is good and you can prove it when most people are trying to sell you "mineral rich" Iron tainted industrial runoff. TDD absolutely can increase costs and complexity in the wrong hands.
The moral of my rambling story is to be Agile I guess. In the original sense, not the consultation services one. To the inexperienced developer I'd say: Try to solve problems, not implement solutions. And above all, never cargo cult. When someone tells you you need 100% coverage, ask them what benefit it provides. When someone tells you to "measure everything", ask them if they've measured the business value of measuring everything and if it outweighed the opportunity cost and dollars sunk into the effort. Be a constant skeptic, because snake oil is everywhere.
Sorry for the diatribe. You took me down memory lane and I guess I feel pretty strongly about TDD.
Could you elaborate?
Speaking for myself, I thought that the main useful thing to come out of BDD was working outside-in and the given/when/then convention.
But I certainly didn't get it on my first pass. Looked at Cucumber, soldiered mightily, gave up in disgust.
"Uncle Bob" Martin has built a lot of impressive things. Most notably FitNesse [1] the service level test framework. He tests his ideas out and is a great teacher.
I've been doing TDD for years thanks to Uncle Bob, Martin Fowler, and Roy Osherove (who have all built impressive things) and I've leveled up as a result. In fact my entire team has leveled up with this simple discipline.
I've yet to meet an anti-TDD zealot who has actually spent a month developing the art of TDD. The detractors tend to be people who write really brittle tests that are a pain to maintain.
[1] - http://FitNesse.org