News for hackers. Stuff that matters.
Re:Giving Up on TDD by msie on Saturday March 19, 2016 @01:19PM (#11319370)
Most software out there has been done without TDD.
Re:Giving Up on TDD by wpietri on Saturday March 19, 2016 @02:51PM (#11319862)
Sure, but most software is also pretty bad. Most software projects aren't considered successful. [1] And 99% of people advocating TDD have written software without TDD, so they've tried it both ways.

Heck, when anesthesia was introduced, people thought it was a needless luxury and an interference with the pain God intended us to have. [2] And they were right in the way you were right: billions had just lived with the pain, so anesthesia wasn't really necessary. But something can be unnecessary and still be a good idea.

[1] https://www.google.com/search?site=&tbm=isch&q=software+proj...

[2] http://www.newyorker.com/magazine/2013/07/29/slow-ideas

Re:Giving Up on TDD by Silhouette on Saturday March 19, 2016 @09:00PM (#11321271)
Sure, but most software is also pretty bad.

That seems rather uncharitable.

Billions of people travel in software-controlled vehicles every day, and most of them will get to their destination safely and reasonably efficiently.

When I check my bank account or credit card statement, it is extremely unlikely that anything on it is incorrect, even though I may have been transacting with other parties all around the world, with several completely different organisations involved along the way.

When I make my dinner, it is a safe bet that all my kitchen equipment will function properly, even though everything from my microwave to my refrigerator is software controlled.

If I pick up my wireless landline handset and place a call, chances are nothing short of severe interference is going to prevent me from connecting to the phone of the person I want to speak to, again even though this might involve intricate negotiations between numerous devices and even different organisations in different countries.

The world is full of software that is actually pretty good considering it was made by fallible humans. We don't notice a lot of the time, because the things depending on that software just work.

Re:Giving Up on TDD by wpietri on Sunday March 20, 2016 @05:34PM (#11324782)
I guess if your criterion for "good software" is "mostly works" then that's fine. Mine are better.

But I have seen bank code. It's terrible. I've seen credit card handling code. It's terrible too. Phones used to be reliable devices, but the average uptime of my phones has fallen to something like 3 days. Perhaps 3x/day my phone tells me that some app has stopped and that it would like to send a bug report. The code that runs cars is terrible too: http://www.safetyresearch.net/blog/articles/toyota-unintende...

Most companies I visit have bug databases with hundreds or thousands of open problems. And that's the just ones they know about, the ones where people take the time to report the bug. Things are terrible and people are just used to it. In the aerospace industry, it's called "normalization of deviance", and it's what destroy the Challenger.

None of this is necessary. It's been at least 15 years that Kent Beck has been talking about the various quality practices that drop bug rates dramatically, including test-driven development, pair programming, and continuous integration. It has been at least 10 years that Martin Fowler has been reporting on teams that have bug rates below 1 per team member per month.

I'm certainly willing to be charitable about the people. A lot of organizations making terrible software are staffed with perfectly nice people who mean well. But I decline to be charitable about the software. These are commercial products, not ash-trays made by second-graders at day camp. The software should stand and fail on its own, with no charity needed to soften the blows.

Re:Giving Up on TDD by Silhouette on Sunday March 20, 2016 @07:29PM (#11325222)
As a slight digression, I think "mostly works" is a reasonable benchmark for a lot of software, because as much as you or I may dislike it, evidently the market won't pay for something qualitatively better. Your phone crashes every few days because most people are willing to accept junkware that crashes every few days even though it's part of one of the most expensive things they'll buy this year. I can't imagine why so many people would think that's acceptable, other than the industry successfully convincing them that it's the best they can reasonably expect, and personally I use a feature phone that doesn't suffer this sort of madness anyway, but sadly it's clear that I'm in a minority here.

If we're talking about software where reliability actually matters, then I agree that too many projects fall far short of ideal standards. I've heard all the same horror stories that probably you have. The auto industry, in particular, is moderately terrifying. But on critical projects, the kind of heavy reliance on unit tests and ad-hoc specification that is common practice and even considered desirable on a lot of Agile projects is also inappropriate, or at least far from sufficient on its own. So I'm assuming we're not really talking about the software that controls a pacemaker, the emergency shutdown systems for a nuclear reactor, or the safeguards to prevent configuring points so trains moving in opposite directions enter the same section of the track.

For most software, though, the programs that help us to do things day-to-day but if they fail once in a while under some awkward conditions it's not the end of the world, I think it's pretty clear that the industry produces a lot of value and the users would miss it if it were gone. Is it perfect? Of course not. There's plenty of room for improvement. But I think the argument that most software is bad is hyperbole.

Re:Giving Up on TDD by wpietri on Sunday March 20, 2016 @11:05PM (#11325948)
> evidently the market won't pay for something qualitatively better

Your assumption here is that quality is more expensive. In my experience, it's substantially cheaper. I've seen "enterprise" shops take reasonably simple apps and blow them up into things requiring large teams and enormous amounts of hardware. And then spend 70% of their time debugging, because they're going too fast to do anything right. This is endemic; some friends of mine do ops consulting, and even at the heart of the tech boom they see clusterfuck after clusterfuck. The apps all mostly work, or the companies would be out of business. But we can do better than just failing to fail.

As an analogy, look at the US car industry in the 70s and 80s. They were producing terrible stuff. Toyota came along and demonstrated you could make better cars for less money. The same opportunity is available here in software. Consider, E.g., WhatsApp, which was serving nearly a billion people on 8 platforms with a team of 50 engineers.

It's an especially appropriate analogy in that a lot of the most effective process improvements come from applying TPS-derived principles to software. See, e.g., Mary Poppendieck's work.

> But I think the argument that most software is bad is hyperbole.

Only if you define bad to mean "worse than average". But I mean it quite literally.

Bug rates, development cost, development cadence, WIP, and project failure rates are all absurdly high compared to well-run projects. This has been true for decades. By "bad" I mean "well below what teams could achieve if they applied best practices".

Re:Giving Up on TDD by Silhouette on Monday March 21, 2016 @06:48AM (#11327218)
Your assumption here is that quality is more expensive.

To some extent, I think it is. More specifically, I think there is a balance between spending more on preventing defects up-front and not needing to spend as much on dealing with those defects later, which dominates the issue up to a certain point, and then beyond that point you have to start considering external costs as the dominant factor.

If you have a project that is made of poorly designed spaghetti, doesn't have any sort of serious test or review processes, and is kept afloat by little more than a few hero developers, then of course you're likely to have a relatively high level of defects. Even modest improvements in the development process will likely have a very good ROI in this case. In this sort of scenario I would agree with you that improving quality may be substantially cheaper than neglecting it, because relatively easy changes in development process would probably pay for themselves in reduced maintenance costs even before considering external factors.

However, the kinds of changes that bring really dramatic improvements in quality -- the kind of thing we might hope you would use for a medical device or safety-critical transport control system -- really can significantly increase development costs. Assuming you could fix the easy problems in other ways, you're probably chasing a relatively small number of extra defects already by the time you get to using these methods. To get a big jump in quality at this point, you might need to employ very different development and/or engineering techniques, such as formal verification stages, redundant systems, or much more structured and demanding review processes, and you might need to do this all the way down your tool chain in both software and hardware terms. These measures tend to require more skills, time and/or resources, and all of those are expensive.

Now, we have to be clear on what we mean by "more expensive". So far, I've mainly been talking about the development costs here, what it takes to write and maintain the software. The point of the extreme quality approaches is usually that failures of the system may have some other cost -- in human life, perhaps, or in delaying something important by a very long time -- that is not acceptable, and so extra investment in avoiding that external cost may be justified even though it makes the development itself much more expensive.

In my experience, the development costs associated with those more extreme approaches ("extreme" is a somewhat loaded term, but I can't immediately think of a better word and I hope you understand what I mean) will be prohibitive today for non-critical software, the kind of system that doesn't have a catastrophic failure case with disproportionate external costs to consider. This is what I mean when I say the market won't pay for something qualitatively better: most people won't prefer to pay $20,000 for a word processor that essentially never crashes or corrupts data or has minor incompatibilities when loading files created using its previous version, instead of $200 for a word processor that basically does its job but might crash out every couple of months and lose the five minutes of work done since the last auto-save.

By "bad" I mean "well below what teams could achieve if they applied best practices".

OK, so if we also restrict "best practices" to "things that improve quality at any cost" then I would agree with you that most software is bad by your definition.

However, if best practices also include things like being commercially viable, then I would no longer agree with the claim that most software is bad by your definition. There certainly is plenty of bad software around, but there's also plenty of software developed in ways that already do avoid silly defects reasonably successfully. In the latter case, I come back to my argument above: because of both diminishing returns in the number of failures you might prevent and the need for more fundamental changes in the development strategy that are relatively expensive to implement if you want to significantly reduce the number of remaining defects, most projects won't be able to do these things with the tools and techniques we have available today and still remain commercially viable. I don't think it's really fair to say those projects aren't well-run just because they went with a strategy that the market would accept.

Re:Giving Up on TDD by wpietri on Wednesday March 23, 2016 @06:42PM (#11348878)
> In my experience [...] will be prohibitive today for non-critical software

And how much time have you spent practicing TDD? Have you worked on a project with 95%+ unit test coverage? Have you worked on with a comprehensive test suite that runs in under 30 seconds? Have you worked in a team that practices pair programming and collective code ownership? Have you worked on a team that does continuous deployment with at least one deployment per developer per day? Have you worked on any team that has bug rates below one per developer per month?

Other people have had different experiences than you. I am one of them. I'm telling you that it's perfectly possible to do an order of magnitude better on bug rates than most teams and get a cost decrease. Plenty of other people will tell you the same. People having been writing about their experiences like this for 15 years.

At this point, I have given up expecting J Random Commenter to believe me; normalization of deviance means that most people cannot (or will not, I can't tell which) even conceive that things could be better. It's the same way that American car companies literally could not understand how Japanese manufacturers were producing radically better products at substantially lower costs. They still generally can't, because to do so would mean admitting that they've been screwing up for decades.

So if you'd like your current limitations, carry on arguing for them. But if you would like to see if something can be different, try out something like Extreme Programming.

Re:Giving Up on TDD by Silhouette on Wednesday March 23, 2016 @11:55PM (#11350469)
It's regrettable how often people assume opinions different to their own must have been formed out of ignorance.

Some of the code I've written has to run in places where the cost of failure can be very high (not normally human life high, but certainly economically prohibitive) and the processes to deploy an update anything if a bug does need fixing can be measured in months with significant costs of their own.

As an example, I wrote a program a while back that implemented somewhat complicated data processing algorithms, took a few months to develop, has been in service for several years now, and to my knowledge has never had a single bug reported against it in production other than a small number where the project met the spec but the spec turned out to have been wrong.

That project was developed and tested using a variety of techniques. A sensible automated test suite was one of them. It was also built on rigorously proven mathematical foundations, among other things.

So yes, I do have experience with building very high quality software. I've made a significant part of my living doing it over the years, and in some cases I have single-handedly outperformed entire teams working for my clients' competitors at the same time. I do know the value of a good test suite, and a lot of other effective development techniques.

It would still be commercially unreasonable to spend the kind of time and money it took to develop a project at that level of robustness if the potential costs of failure were not so high.