News for hackers. Stuff that matters.
Re:The Effectiveness of Test Driven Development (With Case S... by newt on Thursday December 02, 2010 @10:03AM (#1962296)
I'm not sure what you mean by "TDD is the Cause of the bugs" - the TDD projects had 40-90% fewer bugs. Even if their remaining bugs would have been caught by test coverage of things that occur in live use, that's still a good result, right?
Re:The Effectiveness of Test Driven Development (With Case S... by JoeAltmaier on Thursday December 02, 2010 @11:05AM (#1962484)
Well, sort of. TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.

So yes, you record fewer bugs, fix those narrow cases quickly and pretend you're done.

Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.

I vastly prefer shakedown tests of the sort where you launch a 'bot army' on cloud servers to thrash the app. Especially if there is Any network/client/server component.

Re:The Effectiveness of Test Driven Development (With Case S... by newt on Thursday December 02, 2010 @11:11AM (#1962497)
TDD defines bugs as "things the unit tests catch". And the unit tests are very narrow.

I think that if you read the article linked, you'd find that the study didn't work that way. Any study that measures different things between experiment and control groups is not going to be sound.

Then it gets to the field, and whatever you didn't catch becomes a 3-alarm fire.

I've worked on TDD projects, and those are definitely are counted as bugs. Severity 1.

Re:The Effectiveness of Test Driven Development (With Case S... by steveklabnik on Thursday December 02, 2010 @12:44PM (#1962900)
... and then you get a regression test, so that bug never happens again. And the regression test is easy to write, since all of your code is already testable, you have all the testing infrastructure in place, etc.
Re:The Effectiveness of Test Driven Development (With Case S... by JoeAltmaier on Thursday December 02, 2010 @03:07PM (#1963468)
Never happens again? You already have the 3-alarm fire, its too late. That's my point exactly. You catch only what you can think of and write tests to cover.
Re:The Effectiveness of Test Driven Development (With Case S... by newt on Friday December 03, 2010 @08:34AM (#1965713)
I'm not sure how TDD makes this worse. Unless you think that it absolves you of any end-to-end QA testing. In which case, welcome to the fire.
Re:The Effectiveness of Test Driven Development (With Case S... by JoeAltmaier on Friday December 03, 2010 @10:05AM (#1966063)
I object to the TDD fundamental precept that you code to the tests. That narrows the 'testing' until its a self-fulfilling prophecy that you will have 'fewer bugs', since you defined the bugs as 'what we're looking for'.

Reminds me of the old joke "A: I'm looking for my wallet. B: where did you lose it? A: over there. B: then why look here? A: the light's better"

Re:The Effectiveness of Test Driven Development (With Case S... by newt on Wednesday December 08, 2010 @05:23AM (#1982436)
You're repeating your false idea that bugs in TDD come only from the unit tests. Eh, see comments above.