News for hackers. Stuff that matters.
Re:TDD, Straw Men, and Rhetoric by jgon on Wednesday April 30, 2014 @07:07PM (#7677103)
On the one hand this article does at least provide citations for some of the timelines involved. On the other hand it makes me throw my hands up when I see some of the examples provided. 100 lines of code to test 50 lines of code, which implement a catalog is pretty par for the course it seems in TDD advocacy. This could honestly be me venting my current frustrations, but I get tired of this type of advocacy, also often seen in the functional world, of proving how great something is by using the most trivial possible example, in a domain squarely in your technique's wheelhouse. 100 lines of code runs in 0.24s? What is this, I don't even. Your functional algorithm is incredibly elegant at implementing the fibonacci sequence? Awesome.

But are these things honestly the bulk of what people do? Is testing that you can put objects in a catalog, access them, and remove them most of what people do and need assurances on? Am I the only person who has spent most of their career working on software with codebases measured in the millions of lines, with significant user interfaces as well as significant technical domain knowledge embedded in them? I know that TDD says if our objects have many collaborators we are "doing it wrong" but honestly how far do you break down something without it blowing up into a million classes and a ton of code? How do you get around the fact that a button press can set off a numerical simulation (actually these are super testable and I support that), several trips to the database, multiple changes to the user interface, all in the face of dozens of constraints at all levels of the program to ultimately end up with the graphical representation of chemical pressure that the user wants? Especially when every moving piece in that calculation is supposed to be tested apparently in independent units?

I would estimate that the bulk of code that I have experienced in my career relied on multiple other objects to be effective. Do I just mock these out and pay the price of keeping those mocks in sync with the classes they are imitating? Do I then have my test be tightly coupled to the internals of the function being tested, making sure this mock is called in this way, this many times? Do I rewrite things such that every function takes its collaborators as arguments and returns the results I am looking for? What happens when each of those collaborators is itself a giant series of other stateful objects or at least provides access to a highly complex piece of state which is a pain in the ass to setup?

I ask these questions seriously, because on the one hand TDD advocates keep telling me I am not professional if I don't follow their practices, but on the other the details of how to do so in the face of real-world constraints (not 50 line data-containers) are always somehow left out of the discussion. I want to believe, I do, it just gets hard to sometimes.

Re:TDD, Straw Men, and Rhetoric by gary_bernhardt on Wednesday April 30, 2014 @08:12PM (#7677357)
I didn't say "I have a 50 line example, therefore it works in all cases". Destroy All Software is 2,208 lines of production Ruby code, and most of it is tested in exactly that way. I could've showed you charge_purchase_spec, download_policy_spec, etc. The point is that most of the application is decomposed into these little 50 line pieces that can be thought of and tested all by themselves. You look at this and see a trivial example, but the whole idea is that large applications can be built out of lots of trivial little examples and it really, actually works.

You're also misinterpreting what the catalog is. You see one word, "catalog", and decide that all it does is "put objects in a catalog, access them, and remove them". No.

The DAS Catalog class enforces a few integrity guarantees, like "there will be no gaps in serial numbers" and "titles won't be duplicated". Then it provides several querying mechanisms, like finding seasons by slug, finding episodes by slug, or finding the season for a given episode. It also has some aggregation behavior like "give me a list of all screencast titles".

None of those Catalog behaviors is very complex, and all of the tests are simple. That's the point! Most software is made up of fairly simple things like this: aggregate some stuff; find some stuff; check a compound property of something; make a few decisions. These things are easily tested in isolation. For the rest, there's integration testing, or even plain old exploratory testing. But that rest is not nearly as large as it seems to be naively.

The last three paragraphs of your comment misunderstand what TDD is and what it provides you. I don't think that you're "not a professional" if you don't do TDD. I find value in it for a lot of code; so do many other people. Like the article says (you read it, right?), I only do TDD about 75% of the time in web apps, and more like 50% outside of web apps. As is also mentioned in the article (you did REALLY read it, right?), the best way to learn TDD's limitations is to do it 100% of the time, which has the unfortunate property of creating some zealots who haven't yet found a balance.

Re:TDD, Straw Men, and Rhetoric by jgon on Thursday May 01, 2014 @01:30AM (#7678390)
Hi Gary, thank you for the reply. Let me first assure you that I did read your article. I know it's just the word of some guy on the internet but a while back I got so fed up with dumb comments that I promised myself I would never comment on something without having first read it in an honest fashion.

Next, I would say that your comment is still somewhat frustrating to me. My tone may have been overly combative, but I was legitimately asking questions in the hope of getting an answer or at least pointed to one. So I hope you can understand my frustration when you dismiss three paragraphs of my writing as misunderstanding what TDD provides. The may be true, but if you're going to spend time typing those words I feel like you could at least type a few more either taking a stand on what you believe TDD to be or pointing out some resources that I could use to educate myself. As it is you've told me I'm wrong but not how or why.

As for most software being composed of stuff that is simple in the aggregate, I again disagree but this may be my ignorance. Again, most of my experience in software has come on large systems where each object has several collaborators, each of which may also have several collaborators, recursively continuing out. The two options I see for TDD are either to spend considerable effort setting up these collaborators, or to mock them out. If I mock them out, I am coupling myself to the internals of how the method is implemented, because I have to give explicit implementations of each method called in the course of the method doing its job. If the method changes, the mock has to change, and the test breaks, even if the surface behavior of the object may not change.

I may be missing something here, but this is the reality of software as I experience it. Answers that say any of the following do not help me, and honestly I consider any of these evidence that TDD is not cure-all it is made out to be:

1) You're doing it wrong, where wrong is any case TDD has failed, essentially making it unfalsifiable 2) Your system is poorly written and doesn't work for TDD, essentially condemning the vast pool of legacy that makes the world spin to be without tests, or refactored/rewritten without adding user value 3) Your understanding is incomplete, without ever specifying what it would take to have a complete understanding or how to go about gaining this, with a similar symptom as case 1.

I appreciate that you take a more balanced approach to TDD advocacy. I should do a better job of separating you from people like "Uncle Bob" from whom I took the direct quote about not being a professional. I have also dealt with several TDD advocates throughout my career whose responses to questioning TDD have fallen into the previous 3 categories. Because of this I sometimes find it hard to separate more reasoned TDD advocacy from the former, and so my apologies if I misinterpreted the strength of your message.

Re:TDD, Straw Men, and Rhetoric by gary_bernhardt on Thursday May 01, 2014 @03:39PM (#7682227)
The first paragraph of the Wikipedia article on TDD is a correct definition. Write a minimal test; see it fail; make it pass with a minimal code change; refactor to improve the design, keeping the tests passing. There are things that people do alongside TDD; there are common ways to perform TDD. But the red-green-refactor cycle is what "TDD" means.

I don't how many you mean by "several collaborators" in your composition paragraph. I'll assume 8, because that's enough collaborators that TDD will really start to hurt.

All software (not even most, but all) is composed of aggregations of trivially simple components. At the machine level, everything is built from instructions; in a fully OO language, everything is built from method calls; in some functional languages, everything is built from unary functions.

Systems are built by aggregating the trivially simple primitives provided by the environment. We have full control of that aggregation process; it has no mind of its own. We can choose to aggregate four things at a time or eight things at a time. That's not a big difference. It's the difference between 8 and 4 + 4. Same result, different decomposition.

The "4 + 4" analogy is not a straw man: splitting an eight-object interaction into two four-object interactions aggregated together is a well-defined operation on the syntax tree. IDEs even automate it, and have been for a decade or more; this is what an automated "extract method" refactoring is. Doing it well requires years of practice, but all software is made of aggregations of trivially simple components, and we have full control of the aggregation.

Mocking is not required for TDD. It's often used to do isolated unit testing, which may be done within a TDD loop. But even isolated testing can be done without mocks. I did a talk called "Boundaries" about that topic. https://www.destroyallsoftware.com/talks/boundaries

So yes, you are missing something here: first, most people doing TDD aren't mocking, or are mocking very rarely. Second, mocking is not required for isolation; you can also isolate by structuring your software in certain ways, which is what Boundaries is about.

Addressing each of the three responses you anticipate:

1) "Falsifiability": Nothing in software practices is falsifiable in practice. I've never seen a single piece of experimental literature that I considered sound. There's not even experimental evidence saying that "structured programming is better than willy-nilly GOTOs". And I've never even heard of a meta-analysis or an experiment being reproduced several times independently.

Sometimes you really are doing it wrong. If you try Haskell, can't figure out how to write to a file, and throw up your hands, that doesn't mean that Haskell "has failed" or "is unfalsifiable"; it means you don't know how to use it. Haskell and TDD are both particularly difficult to get your head around at first. Maybe you don't want to spend the effort. That's totally fine. This is why I don't actually know Haskell well.

2) "Your system is poorly written." This is a hugely subjective claim and you have to treat it as such. You have to silently append "by my standards of design" to the end of it. You also have to realize that everyone's standards of design are informed by the practices that they've used while doing the design.

If you primarily work on systems composed of functions with, say, ten collaborators (meaning a total of ten arguments and referenced globals/functions/etc.), then yes, I will say "your system is poorly designed". It doesn't mean that I think that you're a bad person; it does mean that I won't work with you to continue building your system in that way. Doing isolated unit testing on that system will be very difficult. Doing integrated unit testing, with or without TDD, will be less difficult. If we crank the collaborators up to 20 or 30, all programming tasks will be difficult, testing or not.

In the second part of your issue (2), you conflate TDD with testing, which is not correct. TDD is a loop of actions that produces tests. There are many other ways to produce tests.

3) "Your understanding is incomplete." If want to understand these ideas, read "TDD By Example" by Beck to learn about the TDD process, then "Growing Object-Oriented Software Guided by Tests" by Freeman and Price to learn about TDD design feedback and the careful use of mocks. Yes, it'll take time. (But less time than learning Haskell.)

If you want to understand what I mean about isolation without mocks, watch my "Boundaries" talk (I'd recommend doing that after reading the two books above). If you want to see live examples of TDD, and the trade-offs inherent in TDD being made and dicussed, watch my Destroy All Software screencasts.

However, if you don't want to do the work to tease these ideas apart, then I think that you should acknowledge that you're not willing to put the effort in. This is a different path than saying "people haven't said the right things to me for me to believe it works", which is the vibe I get right now. I learned TDD by doing it, incorrectly, and painfully, over and over again. You have the advantage of being able to read a couple of books to jump past my first year of learning. That's a huge efficiency gain, but the process can't be compacted into a comment on Hacker News that transmits the better part of a decade of experience.

(Finally, somewhat tangentially, I recommend that all programmers disabuse themselves of any belief that we have experimental evidence about programming practices by reading "The Leprechauns of Software Engineering".)

Re:TDD, Straw Men, and Rhetoric by Perceptes on Thursday May 01, 2014 @06:21AM (#7679033)
You bring up some very valid concerns. I'm an advocate of TDD, but I'd love to hear some responses from TDD advocates on the disconnect you see between the TDD ideal and the reality you experience, preferably in a less defensive and accusatory tone than Gary's reply to you.
Re:TDD, Straw Men, and Rhetoric by jshen on Wednesday April 30, 2014 @08:53PM (#7677509)
Know of any open source code bases built this way?
Re:TDD, Straw Men, and Rhetoric by antrix on Wednesday April 30, 2014 @07:41PM (#7677241)
As a start, I recommend reading "Growing Object-Oriented Software, Guided by Tests". I am still not a 100% convert but I did get many of the same questions answered.

http://www.amazon.com/Growing-Object-Oriented-Software-Guide...

Re:TDD, Straw Men, and Rhetoric by jgon on Wednesday April 30, 2014 @07:43PM (#7677248)
Can you believe that I literally bought that book yesterday? I guess I have even more motivation to read through it now!
Re:TDD, Straw Men, and Rhetoric by currywurst on Wednesday April 30, 2014 @07:57PM (#7677298)

  > TDD advocates keep telling me I am not professional
Well it's nice that they hold the keys to the kingdom ;) ! In your darkest hours, remember this powerful mantra: "There is no silver bullet", and proceed to systematically take down glib generalizations.