News for hackers. Stuff that matters.
Re:Is TDD Worth It? by anon1385 on Saturday September 28, 2013 @09:18AM (#6461736)
TDD won't help you solve problems that you don't already know how to solve: http://ravimohan.blogspot.co.uk/2007/04/learning-from-sudoku...

In fact it only gets in your way. Solving problems that are new (to you) involves writing code to find out if a certain approach works, then throwing it away and trying something else. TDD makes that process a lot slower, and makes people more reluctant to throw away incorrect models of the problem that they have codified into tests without even realising it.

Of course somebody is going to chip in here and say that if the above is happening then you are writing your tests at the wrong level of abstraction and that you should be testing at the point where you can create a stable API that remains unchanged while you throw away the speculative implementations underneath. But if you are tacking a new problem you have no idea where those boundaries lie. It's hard enough to determine that for problems that you already understand and have working solutions for, never mind ones you don't. TDD forces you to decide about one of the hardest parts of development (API design / where to place abstraction boundaries) up front before you have worked out how to solve the problem at all. Either that or you just write vast vast numbers of trivial tests for every single function that you write (which does seem to be what some TDD tutorials advocate), but then you are creating a huge maintenance burden for the future where any change to the code is going to necessitate rewriting dozens of tests.

I would argue that TDD only works if you are solving problems that you are very familiar with. By the time you are writing your 5th CRUD web app you probably have decent mental model of the parts you need before you start. I feel that a lot of the love for TDD comes from people who spend most of their time working on problems they already solved dozens of times before, but they learned how to solve them before they discovered TDD.

For people first learning how to code it is absolute death. It discourages people from doing the most important thing they need to learn: throwing code away.

Re:Is TDD Worth It? by alinajaf on Saturday September 28, 2013 @09:41AM (#6461782)
> TDD won't help you solve problems that you don't already know how to solve:

I talk a lot about web application security. The following line from flesh and blood developers on multiple occasions:

"Shouldn't your tests catch security vulnerabilities?"

To which I always respond:

"Yeah sure, they'll catch the vulnerabilities that you already thought of"

I've personally seen codebases with test suites that cost (even with the most conservative estimates of time spent and day rate) upwards of 300,000 USD with 100% coverage that allowed an anonymous client to drop the users table.

If a lack of security bugs (and therefore bugs in general) is any indicator of software quality, TDD has yet to impress me in the wild.

My development process has always been to prototype out three or four implementations, evaluate them against the known use cases and then settle on a final API design. Then I'm ready to do TDD. Glad to know I'm not the only one who works roughly this way.

It also bugs me a little that many otherwise intelligent developers use TDD as their sole software design tool much of the time. The exact term they use for this is using tests to "drive out" a design. While I understand this may work in textbook examples, in real life I've seen the concept being used to justify designs that clearly aren't fit for purpose.

Re:Is TDD Worth It? by InclinedPlane on Saturday September 28, 2013 @10:15AM (#6461850)
That illustrates a very serious problem with TDD, the idea of coverage.

100% code coverage is not 100% coverage of every different possible code path. In fact, for most software systems it is not even remotely feasible to achieve that level of code path coverage. The "100%" figure can easily lull less experienced devs into thinking they have sufficient test coverage when nothing could be further from the truth.

Re:Is TDD Worth It? by mbrock on Saturday September 28, 2013 @10:28AM (#6461878)
However, TDD encourages – ideally – a style of programming where 100% branch coverage is likely to mean the program is correct.

That's part of the idea of the "unit" in "unit test." You decompose the program into units for which correctness is relatively simple – and testable.

How to do this properly is not obvious, and for teams on which TDD is imposed without this understanding, it will all become a bizarre and useless ideology.

When it is understood, the resulting program will have properties that resemble those of purely functional programs, especially composability and correctness.

Re:Is TDD Worth It? by seanmcdirmid on Saturday September 28, 2013 @11:31AM (#6462018)
> when it is understood, the resulting program will have properties that purely functional programs, especially composability and correctness.

It is not that hard to write pure functional code that is not very compassable or correct. Keeping it simple stupid is more important.

Re:Is TDD Worth It? by amorphid on Saturday September 28, 2013 @12:30PM (#6462228)
This isn't a problem with TDD. It's a problem with the developers using it. TDD doesn't protect you from being a moron.
Re:Is TDD Worth It? by haar on Saturday September 28, 2013 @09:31AM (#6461758)
If you were to ask anyone in my office, you'd hear how much I praise TDD as a software development practice. However, when I'm absolutely clueless as to how to even begin to tackle a technical problem (usually due to not enough understanding of the overall system requirements), then I start by spiking the problem - trying to get a feel as to everything that's required. This initial spiking I wouldn't test-drive, as it's a learning process and not production-ready code.

After I've got a strong enough grasp as to the overall requirements, then I throw away this spiked code to start fresh, working along similar lines to 'programming by wishful thinking' whilst keeping the original spiked out structure in mind.

By following a strong TDD philosophy, it's easier for me to throw away my initial 'spiked out' code base for a couple of reasons:

1. Personal experience of how much better a test-driven code base can be than ones I've later tried to wrap tests around/dismantled in the process. 2. Having the understanding from the get-go that this code is not 'production-ready', and should not be considered reliable, but as a very fast learning experience. 3. Knowing that I've spent a small time prototyping as preparation, and having this laid out as an overall part of the process means I'm not considering this time 'wasted'.

Re:Is TDD Worth It? by yummyfajitas on Saturday September 28, 2013 @09:56AM (#6461811)
When tackling a new problem, you still need to build a lot of utility functions that you know how to write. Tests help here.

For example, I'm building a statistical model. I need to load a bunch of data from files, transform it in straightforward ways, then run my model against it. TDD doesn't help me come up with the right model, but it does help me not screw up while loading the files. When my model fails to be predictive, I know the model (rather than the loader) is at fault.

Re:Is TDD Worth It? by drewhk on Saturday September 28, 2013 @09:39AM (#6461777)
Absolutely! In the past I worked on an image recognition project where I implemented, tried and thrown away 14 distinct algorithms/approaches (not counting variants!) for a particular problem. TDD is unfit for this because you tend to commit yourself to the set of tests instead of throwing all of them out (all of the approaches were fundamentally different and I had soft real requirements -- so optimized representations were a must). Of course we had some unit tests, but not written in TDD style since the goal was to find the correct approach -- not the correct implementation.
Re:Is TDD Worth It? by jasonlotito on Saturday September 28, 2013 @09:47AM (#6461795)
If you are trying to do 14 different things, I don't know why you'd expect TDD to solve that for you. That's a different problem.
Re:Is TDD Worth It? by drewhk on Saturday September 28, 2013 @09:54AM (#6461807)
Not 14 different things. 14 different approaches to solve the same image recognition problem. Please do not jump to conclusions. I did not say TDD is not useful at a _later_ stage, but at the research phase it is simply more of a burden than help, because the "refactor" step ends up as "drop everything".
Re:Is TDD Worth It? by jasonlotito on Saturday September 28, 2013 @10:38PM (#6463886)
> Not 14 different things. > 14 different approaches

So... if you are trying to take 14 different approaches... TDD isn't there to solve it for you. That's what I said.

But then that's planning. And when you've found your approach, you write it with TDD.

Re:Is TDD Worth It? by jasonlotito on Saturday September 28, 2013 @09:45AM (#6461790)
> TDD won't help you solve problems that you don't already know how to solve

So, those Sudoku testing article fails because the author is breaking rules of TDD. I mean, the first test: test_cells, does so much more than it needs to do to pass the test. He's designing first, and then testing. That's not how TDD works.

If you can't see why that first test is a bad test, you aren't qualified to judge the merits of TDD.

> Of course somebody is going to chip in here and say … you should be testing at the point where you can create a stable API that remains unchanged

That's not true. At least, not the way you phrase it. You are, of course, forgetting that TDD involves 3 steps. Writing a test, Passing that test, and then refactoring. That refactoring includes refactoring the test. API's can, and should change to evolve. That's part of refactoring.

You pretty much repeat the same arguments against TDD that I see time and time again. Arguments born out of misunderstandings. TDD isn't easy. It's challenging, it requires practice and effort. It does take time.

> TDD forces you to decide about one of the hardest parts of development (API design / where to place abstraction boundaries) up front before you have worked out how to solve the problem at all.

Yes, but nothing about TDD prevents you from easily making changes as needed. Again, Refactoring is apart of TDD. You say something like this multiple times, making it seem like refactoring or changing code is some painful process.

It's not. It's trivial. If it's not, your process has a serious problem.

Re:Is TDD Worth It? by lifeisstillgood on Saturday September 28, 2013 @10:14AM (#6461849)
> That's not how TDD works

if you are looking at Ron Jeffries Doyen of the Agile set, and saying "well of course he did not get the results, he was doing it wrong" the we are well into id-protecting, religious territory.

if one of the leading exponents gets the process wrong there is something wrong with applying the process to the real world

TDD really seems to me to be two parts:

- Design up front, disguised as writing an API

- Ensuring you write repeatable tests before commuting

Re:Is TDD Worth It? by dragonwriter on Saturday September 28, 2013 @03:12PM (#6462784)
> if one of the leading exponents gets the process wrong there is something wrong with applying the process to the real world

I'm more interested in extracting what works from a concept than slavishly following a cult leader. IME, being a "leading exponent [sic]" of anything doesn't make you infallible, even in that domain.

> Design up front, disguised as writing an API

The TDD literature I am familiar with doesn't advocate writing an API up front, it recommends writing essentially a minimal fact statement up front about the API, writing tests to verify that fact statement, writing code to pass that test if it doesn't already pass with the existing code base, and iterating like that. You could design more than the single fact that you make the test and code for at one time, but that's not a demand of TDD or even relevant to it, and is not, in most cases, consistent with the kind of decide-as-late-as-possible rule that the Agile and Lean approaches in which TDD is usually embedded prefer. I suppose it would make some sense if you were doing waterfall-adapted-to-incorporate-TDD, but why on earth would you do that?

Re:Is TDD Worth It? by jasonlotito on Saturday September 28, 2013 @10:39PM (#6463888)
Anyone who talks about TDD as "two parts" doesn't know TDD.

Wouldn't worry about it.

Re:Is TDD Worth It? by dragandj on Saturday September 28, 2013 @10:41AM (#6461899)
I didn't check with the literature, but in my opinion, the biggest misunderstanding of TDD is that lots of people take it to mean literary Test-First Design. As I understanding, in Test-DRIVEN-Design, you are writing the implementation code in PARALLEL to testing code. You write a tiny bit of test, then a tiny bit of implementation to satisfy the test. If you notice something you'd missed previously, you refactor BOTH tests and implementation. You repeat and expand a bit...
Re:Is TDD Worth It? by RogerL on Saturday September 28, 2013 @11:40AM (#6462043)
Read the literature. Kent Beck, the author of the idea and the book describing it, very clearly argues for test first. The test tells you how to design. I'm not sure if that is what you mean by your paragraph or not. But in his book, the test is always written first, and it guides you into how you should write your code.

I found it extremely dubious. He was doing some kind of accounting/banking app, and came up with a new approach. Well, he spent his career on this domain. I find it far less likely that TDD caused the new idea than it was just that he'd hoed this row so many times before. If you have done it 10 times before you can do bottom up, top down, inside out, outside in, or completely random, and get a reasonably structured, working code base. I conclude nothing by him creating accounting app #11 with a scheme that is perhaps well suited to how his brain works, but is perhaps terrible for mine.

I'm not against testing, quite the contrary, but I find some of the claims rather exuberant.

Re:Is TDD Worth It? by jasonlotito on Saturday September 28, 2013 @10:27PM (#6463867)
> Read the literature. Kent Beck, the author of the idea and the book describing it, very clearly argues for test first.

No, you are misunderstanding the parent. He meant some people assume you write _all_ tests first. If you read his entire comment, you'll see how he describes exactly the process described by Kent Beck. A small test, a small implementation, followed by refactoring both test and implementation (the 3rd part being the part commonly forgotten).

Re:Is TDD Worth It? by jasonlotito on Saturday September 28, 2013 @10:26PM (#6463862)
> You write a tiny bit of test, then a tiny bit of implementation to satisfy the test. If you notice something you'd missed previously, you refactor BOTH tests and implementation.

Bingo! Exactly that. Right on the money, especially the part about refactoring both tests and implementation.

Re:Is TDD Worth It? by recursive on Saturday September 28, 2013 @11:07AM (#6461958)
If you didn't check the literature and have your own understanding of what TDD is, then it should be no surprise that what you are talking about is different from what everyone else is talking about.
Re:Is TDD Worth It? by jasonlotito on Saturday September 28, 2013 @10:28PM (#6463871)
This is what the parent said:

> You write a tiny bit of test, then a tiny bit of implementation to satisfy the test. If you notice something you'd missed previously, you refactor BOTH tests and implementation.

That's TDD.

He was alluding to the fact that people might assume that they have to write all the tests first.

Re:Is TDD Worth It? by mkolodny on Saturday September 28, 2013 @11:40AM (#6462044)
She/he is talking about exactly what everyone else is talking about. That's TDD. "Didn't check the literature" may have meant that the reply didn't come after reading the referenced book.
Re:Is TDD Worth It? by jasonlotito on Saturday September 28, 2013 @10:24PM (#6463860)
He was wrong. There's nothing about it.

Are you really going to sit their and say that he couldn't have been wrong? That he's perfect? That he can't have made a mistake.

The first test in that blog post was a bad test to start with. It's as simple as that. I'm sorry, but appealing to authority isn't going to work here.

He was wrong.

Re:Is TDD Worth It? by majelix on Saturday September 28, 2013 @09:47AM (#6461796)
Yes, if you can't solve a problem, you certainly can't solve a problem in a supportable manner.

Agile has the concept of Spikes (http://www.scaledagileframework.com/spikes/) for researching new problems. The key point that code is thrown away after completion -- this gives you the freedom to ignore normal methodologies and not have to worry about technical debt.

Presumably, you then go back with this new knowledge and only now concern yourself with writing supportable code.

Re:Is TDD Worth It? by giulianob on Saturday September 28, 2013 @09:38AM (#6461773)
From my experience, I've found that TDD is mostly valuable where writing the test will make writing the code more efficient. A lot of the times its because manually testing it to get the code right will add significant amount of time and then you will still need to unit test anyways. If I don't know exactly what I need to do, sometimes I will write the code to get a proof-of-concept working then just comment it out. Then I write the tests and uncomment the code as the tests are written to make sure I have proper coverage and I haven't missed anything.

I hate writing tests that require me to do any kind of boilerplate code. Tests should (theoretically) only set up and exercise whatever is needed for that test. I think if you are able to do that (I've had success with AutoFixture for .NET as an example) then you'll feel a lot less bad about throwing away tests and they will also be much much faster to write. When writing tests is tedious or brittle, then I feel they are a chore and am less likely to write them to begin with.

Re:Is TDD Worth It? by trothoun on Saturday September 28, 2013 @09:39AM (#6461776)
sometimes you can't even really express what the problem is, until you've taken a stab a solving it. The problem might be something like 'This feature is awkward and doesn't feel quite right.' When I'm doing exploratory coding, I need to be able to feel my way through a problem. I have yet to learn to do that from the vantage of tests, and am somewhat skeptical that I ever will.
Re:Is TDD Worth It? by RougeFemme on Saturday September 28, 2013 @05:29PM (#6463215)
You can't really express it, but you still have an idea of what the problem is. . ."solving it". . .you have an "it" in your mind. Also, in many cases, when you are working on a "less technical" feature, there are others who can help express what the problem is. . .user, designer, architect, etc. It's often really tough for a developer to express what the problem is in a vacuum.
Re:Is TDD Worth It? by mkolodny on Saturday September 28, 2013 @11:49AM (#6462066)
I was taught that you should solve the problem first, and then write the code. TDD doesn't change that; it forces you to at least define the problem before you take a stab at writing the code.

Of course, as you delve deeper into the problem, you often learn something that changes your understanding of the problem. That's where the third step of TDD comes in - refactor. TDD doesn't make people reluctant to refactor their code; it forces them to do it. Refactoring is a vital part of TDD.

Re:Is TDD Worth It? by seanmcdirmid on Saturday September 28, 2013 @12:28PM (#6462220)
> I was taught that you should solve the problem first, and then write the code

It is sad that in this day and age waterfall is still taught as a serious methodology.

Re:Is TDD Worth It? by ollysb on Saturday September 28, 2013 @10:51AM (#6461918)
TDD does allow you scope for exploration work. If you watch Kent Beck's TDD screencasts[1] you'll see him follow a workflow where he's exploring an unfamiliar protocol. What's interesting is that he first frames his exploration within a particular test e.g. delete item using protocol, and then writes implementation details inside the test itself until he figures out how it works. He then refactors the test so that the implementation code is moved into the driver that he's developing and the test itself becomes a proper spec. He's working in java but I tend to follow this workflow in ruby. With ruby you can fire up a REPL in the middle of your test code and gradually add to the test as you figure things out.

http://pragprog.com/screencasts/v-kbtdd/test-driven-developm...

Re:Is TDD Worth It? by benihana on Saturday September 28, 2013 @11:22AM (#6461991)
>In fact it only gets in your way. Solving problems that are new (to you) involves writing code to find out if a certain approach works, then throwing it away and trying something else. TDD makes that process a lot slower, and makes people more reluctant to throw away incorrect models of the problem that they have codified into tests without even realising it.

This is a giant strawman. Who is saying to write tests for your prototypes? You seem to be confusing a bunch of different methodologies here. You're talking about rapid prototyping to understand a domain before building a long term solution. That isn't TDD, and it would be foolish to write tests for an exploratory exercise that you're planning to throw away.

>TDD forces you to decide about one of the hardest parts of development (API design / where to place abstraction boundaries) up front before you have worked out how to solve the problem at all

How do you figure? If you've done a rapid prototype like you're advocating, you should have a better understanding of where boundaries lie and how the API should work. Further, you seem to be working under the impression having tests make it hard to change things. If anything, I've noticed the opposite - tests make it much easier to change code, especially when you test behavior, not implementation.

Re:Is TDD Worth It? by sambeau on Saturday September 28, 2013 @12:40PM (#6462262)
If you are iterating their isn't always distinct prototypes — just quick first-draft implementations. Sometimes these are good enough.

In my experience once everyone is happy with the way something works then it is sensible to move to lock-down — tests, documentation, performance testing, refactoring.

Doing any of those four things is a waste of time and energy until the design is decided upon. In agile development the design is not decided upon before the first code is written. It is always jointly discovered through iteration and discussion between designers, user-testers and developers.

If you write your documentation first then I suspect you can get away with writing all your tests first. But that's not a process I would call 'agile' and I doubt it ever results in good software.

Re:Is TDD Worth It? by halostatue on Saturday September 28, 2013 @12:34PM (#6462246)
I interviewed with a company one time that ran with TDD and pairing like a religion.

The guy I was interviewing with didn't like the fact that I thought about how the problem might be shaped (and therefore made assumptions as to what I might want to implement and refine via tests) prior to “writing” that first failing test. It was absolute nonsense and I was rather shocked that the company managed to ship anything, to be honest.

I'm a big fan of tests; I'm a big fan of pairing. They are tools; nothing more.

Re:Is TDD Worth It? by rrouse on Saturday September 28, 2013 @12:51PM (#6462309)
An interview recently felt like that. I was just throwing together little bits of code to make sure I understood the problem clearly, but the interviewer seemed to be a bit agitated that I didn't write a test right away.

I honestly don't work like that. So I'm kinda glad I failed that interview.

Re:Is TDD Worth It? by halostatue on Saturday September 28, 2013 @01:05PM (#6462349)
Yeah. I've recently done a couple of interviews on the other side of the fence (as the interviewer), and I've hit on a problem that can be worked out collaboratively between the interviewee and the interviewer—and how far we go depends on their skill.

It always begins a discussion of the problem and what we want to solve, plus a bit of how—so we can figure out what it is that we're going to be testing for and how to build from that point.

Re:Is TDD Worth It? by ajross on Saturday September 28, 2013 @11:47AM (#6462061)
I think the point is more the the final "prototype" is the "implementation". The core fallacy of TDD is that there is a single moment where you start work. True iterative development doesn't work that way.

Which is not to say that both TDD and iteration don't have their uses. But they work together poorly. And the problems best solved by TDD seem boring to some of us.

Re:Is TDD Worth It? by marxidad on Saturday September 28, 2013 @01:02PM (#6462342)
TDD is supposed to work WITH iteration. You have one feature, you write the test for it, you implement it, pass the test, and go on to the next feature, etc.
Re:Is TDD Worth It? by ajross on Saturday September 28, 2013 @01:26PM (#6462428)
Implementing features piecewise is not remotely the same thing as iterative development. The point to iteration is that you develop your understanding of the problem along with the solutions, many of which get thrown away.