News for hackers. Stuff that matters.
Re:How deep are your unit tests? by philbo on Monday December 10, 2012 @04:02AM (#4898173)
I suspect there are some subtleties to that first answer which you've overlooked.

It is written by Kent Beck. He pretty much invented the concept of test-driven development. He's probably the most well-known advocate of developers writing their own tests there is.

What his answer demonstrates (to me) is that the conclusions he and the other XP guys arrived at - to write code in tiny chunks, each one preceded by a tiny failing test - is based entirely in pragmatism. He does it because it works better than anything else he's come across, including leaving the tests up to a dedicated QA team.

Re:How deep are your unit tests? by dalke on Monday December 10, 2012 @06:46AM (#4898586)
"... because it works better..."

Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html: "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile."

There is very little research which shows that TDD is better than other approaches. Perhaps a bit, for some cases, but not enough to be certain about its general applicability. (Or has this changed in the last three years since I researched this topic?)

I hate writing code in "tiny chunks, each one preceded by a tiny failing test." I prefer to use a small set of burn tests until I have the main structure in place. Only then do I do the tiny tests - through the high-level API where possible (so I can refactor later) - and I use coverage analysis to make sure I'm not missing any obvious tests.

Coverage analysis is not part of TDD, which is a shame. (Technically that's because TDD is a design methodology, while coverage analysis is part of testing. While TDD generates tests, these are not a complete set of tests. For example, security analysis and scalability are outside the scope of TDD.)

Re:How deep are your unit tests? by stonemetal on Monday December 10, 2012 @07:27AM (#4898714)
Except that it hasn't been shown to work better. Quoting from http://third-bit.com/blog/archives/4529.html: "If you ask its advocates for evidence, they’ll tell you why it has to be true; if you press them, you’ll be given anecdotes, and if you press harder, people will be either puzzled or hostile."

Oh good, evidence against is just as preachy and unscientific as the evidence for.

http://evidencebasedse.com/?q=taxonomy/term/2

Research suggests that unit testing has a very strong correlation with correct software. Research suggests TDD has no correlation with correct software, except for the fact that TDDers tend to write 2x more tests than non TDDers.

Re:How deep are your unit tests? by pekk on Monday December 10, 2012 @08:37AM (#4898882)
Your requirement of 'evidence against' isn't how science works. The burden of evidence rests with the people making the claim. If you make a claim and do not provide evidence on request, that claim is not supported; it is not necessary to supply 'evidence against' for the claim to be unsupported, and there's no good reason to believe unsupported claims.
Re:How deep are your unit tests? by stonemetal on Tuesday December 11, 2012 @08:41AM (#4904443)
This is a discourse not science class both sides of the argument should have evidence.
Re:How deep are your unit tests? by dalke on Tuesday December 11, 2012 @10:03AM (#4904740)
Who says there are only two sides? There are dozens or hundreds. I can make a new one - programming is best done with a candle burning on the table and a glass of wine or whisky available. It works for me, so I say it should work for everyone. I call it "Romantic Driven Programming" or "RDP." Now, what's the evidence that you have against RDP as a best practice?

You don't have any, do you? So obviously I can now promote it as the new universal methodology.

(Or more directly, you think the argument is that there is evidence for and evidence against TDD. My argument is that there isn't enough evidence to make a conclusion.)

Given that this is a discourse, have you read any of the discussions of the topic of TDD? For example, "Making Software" (e.g Oram and Wilson) has an excellent chapter summarizing the research results. If not, then it's a rather unbalanced discourse, no? You pointed to a list of papers, but didn't describe what you wanted to discuss from them.

For example, some of the research compares test-first with test-last. I'm a proponent of test-during, and don't think that test-first and test-last are the only two ways to compare things.

Re:How deep are your unit tests? by jiggy2011 on Monday December 10, 2012 @08:40AM (#4898894)
I'm not sure how one would do such research.

You would need to compare teams of equal skill level who were working on very similar software over quite a long period of time.

What would the benchmarks be? Number of defects in production code? Time to ship? Severity of defects? Time spent debugging?

Re:How deep are your unit tests? by dalke on Tuesday December 11, 2012 @10:19AM (#4904798)
It's not easy research, but there have been some papers published on the topic. The problem of course is that there are a lot of confounding factors.

Besides, that my point, isn't it? Promotors of TDD argue that their personal intuition suffices, despite having little evidence to back their claims.