This is exactly how I feel about TDD, but it always feels like you're not supposed to say it. Even in environments where features are described, planned, designed, refined, written as ACs, and then developed, there are still almost always pivots made or holes filled in mid-implementation. I feel like TDD is not for the vast majority of software in practice - it seems more like something useful for highly specialist contexts with extremely well defined objective requirements that are made by, for, and under engineers, not business partners or consumers.

That rings true to my experience, and TDD doesn't add much to that process.

I've advocated the same for early projects or new features. Dump something bad and flawed and inefficient, but which still accomplishes what you want to accomplish. Do it as fast as possible. This is your vomit draft.

I strongly believe that the amount a team could learn from this would be invaluable and would speed up the development process, even if every single line of code had to be scrapped and rebuilt from scratch.

An executing program is a lot less forgiving, for obvious and unavoidable reasons.

What TDD brings to the table when you are building a throwaway version is that it helps to identify and deal with the things which are pure implementation errors (failure to handle missing inputs, format problems, regression failures and incompatibilities between different functional requirements). In some cases it can speed up the delivery of a working prototype, or at least reduce the chance that the first action that the first non-developer user of your system does causes the whole application to crash.

Genuine usability failures, performance issues and failure to actually do what the user wanted will not get caught by TDD, but putting automated tests in early means that the use of the prototype as a way of revealing unavoidable bugs is not downgraded by the existence of perfectly avoidable ones. It may also make it easier to iterate on the functionality in the prototype by catching regressions during the changes cycle, although I'll admit that the existence of lots of tests here may well be a double-edged sword. It very much depends on how much the prototype changes during the iterative phase, before the whole thing gets thrown away and rebuilt.

And, when you come to building your non-throw away version, the suite of tests you wrote for your prototype give you a check list of things you want to be testing in the next iteration, even if you can't use the code directly. And it seems likely enough that at least some of your old test code can be repurposed more easily than writing it all from scratch.

Isn't there some law as well that states that successful software projects start with a working prototype, and software designed from the ground up is destined to fail?

> In most projects, the first system built is barely usable. It may be too slow, too big, awkward to use, or all three. There is no alternative but to start again, smarting but smarter, and build a redesigned version in which these problems are solved. The discard and redesign may be done in one lump, or it may be done piece-by-piece. But all large-system experience shows that it will be done. Where a new system concept or new technology is used, one has to build a system to throw away, for even the best planning is not so omniscient as to get it right the first time.

For more junior engineers, I think TDD can help, but once you "master" TDD, you can throw it out of the window: "clean code" will come out naturally without having to write tests first.

But it got me thinking about testing while writing the class instead of shoehorning a class into a test just before the PR went up. That’s what I think my takeaway was. To this day I think about not just how clean/maintainable the code is, but also how testable the code is while I am writing it. It really helps keep monkeypatching and mocking down.

“Write it three times. The first time to understand the problem, the second time to understand the solution and the third time to ship it.”

The only other clue I have is that it was apparently someone who was super productive and wrote a ton of the early common Unix tools.

Doesn't matter how good you are the v1 of a program in an unknown area is always complete crap but lets you write an amazing v2 if you paid attention making v1.

(Only slightly joking)

An implementation without definition, and a whole host of assumptions gets delivered as v1.

Product expectations get lowered, bugs and defects raised, implementation is monkey patched as v2.

devs quit, managers get promoted, new managers hire new devs, they ask for the definition and they're asked to follow some flavor of the year process (TDD, Agile, whatever).... rinse and repeat v3.

Sad. True. Helpless. Hopeless.

It’s always possible to write a test case that covers a new high-level functional requirement as you understand it; part of the skill of test-first (disclaimer - I use this approach sometimes but not religiously and don’t consider myself a master at this) is identifying the best next test to write.

But a lot of people cast “unit test” as “test for each method on a class” which is too low-level and coupled to the implementation; if you are writing those sort of UTs then in some sense you are doing gradient descent with a too-small step size. There is no appreciable gradient to move down; adding a new test for a small method doesn’t always get you closer to adding the next meaningful bit of functionality.

When I have done best with TDD is when I start with what most would call “functional tests” and test the behaviors, which is isomorphic to the design process of working with stakeholders to think through all the ways the product should react to inputs.

I think the early TDD guys like Kent Beck probably assumed you are sitting next to a stakeholder so that you can rapidly iterate on those business/product/domain questions as you proceed. There is no “upfront spec” in agile, the process of growing an implementation leads you to the next product question to ask.

In my experience, the best time to do "test for each method on a class" or "test for each function in a module" is when the component in question is a low level component in the system that must be relied upon for correctness by higher level parts of the system.

Similarly, in my experience, it is often a waste of effort and time to do such thorough low level unit testing on higher level components composed of multiple lower level components. In those cases, I find it's much better to write unit tests at the highest level possible (i.e. checking `module.top_level_super_function()` inputs produce expected outputs or side effects)

And then, property tests are more likely to be possible, and IMO should be preferred!

test driven re-design?

Definitely agree with you here - I've seen people dogmatically write unit tests for getter and setter methods at which point I have a hard time believing they're not just fucking with me. However, there's a "sweet spot" in between writing unit tests on every single function and writing "unit tests" that don't run without a live database and a few configuration files in specific locations, which (in my experience) is more common when you ask a mediocre programmer to try to write some tests.

Those tests suit a project that applies the open-closed principle strictly, such as libraries / packages that will rarely be modified directly and will mostly be used by "clients" as their building blocks.

They don't suit a spaghetti monolith with dozens of leaky APIs that change on every sprint.

The harsh truth is that in the industry you are more likely to work with spaghetti code than with stable packages. "TDD done right" is a pipe dream for the average engineer.

I always suspect that many people who have a hard time relating to TDD already have experience writing these class & method oriented tests. So they understandably struggle with trying to figure out how to write them before writing the code.

Thinking about tests in terms of product features is how it clicked for me.

That being said, as another poster above mentioned, using TDD for unstable or exploratory features is often unproductive. But that’s because tests for uncertain features are often unproductive, regardless if you wrote them before or after.

I once spent months trying to invent a new product using TDD. I was constantly deleting tests because I was constantly changing features. Even worse, I found myself resisting changing features that needed changing because I was attached to the work I had done to test them. I eventually gave up.

I still use TDD all the time, but not when I’m exploring new ideas.

You can (and often should!) have a suite of unit tests, but you can choose to write them after the fact, and after the fact means after most of the exploration is done.

I think if most people stopped thinking of unit tests as a correctness mechanism and instead thought of them as a regression mechanism unit tests as a whole would be a lot better off.

Also a passing unit test always provides up-to-date implicit documentation on how to use the tested code.

Writing before the fact allows you to test your mental model of the interface, unspoiled by having the implementation fresh in your mind. (Not entirely, since you probably have some implementation ideas very early on.)

Writing tests after the fact is what you must do to explore 1) weak points that occur to you as you implement, and 2) bugs. After-the-fact testing also allows you to hone in on vagueness in the spec, which may show up as (1) or (2).

But TDD is fantastic for growing software as well! I managed to save an otherwise doomed project by rigorously sticking to TDD (and its close cousin Behavior Driven Development.)

It sounds like you're expecting that the entire test suite ought to be written up front? The way I've had success is to write a single test, watch it fail, fix the failure as quickly as possible, repeat, and then once the test passes fix up whatever junk I wrote so I don't hate it in a month. Red, Green, Refactor.

If you combine that with frequent stakeholder review, you're golden. This way you're never sitting on a huge pile of unimplemented tests; nor are you writing tests for parts of the software you don't need. For example from that project: week one was the core business logic setup. Normally I'd have dove into users/permissions, soft deletes, auditing, all that as part of basic setup. But this way, I started with basic tests: "If I go to this page I should see these details;" "If I click this button the status should update to Complete." Nowhere do those tests ask about users, so we don't have them. Focus remains on what we told people we'd have done.

I know not everyone works that way, but damn if the results didn't make me a firm believer.

Unit tests are still easy to write but most complex software have many parts that combine combinatorially and writing integration tests requires lots of mocking. This investment pays off when the design is stable but when business requirements are not that stable this becomes very expensive.

Some tests are actually very hard to write — I once led a project that where the code had both cloud and on-prem API calls (and called Twilio). Some of those environments were outside our control but we still had to make sure they we handled their failure modes. The testing code was very difficult to write and I wished we’d waited until we stabilized the code before attempting to test. There were too many rabbit holes that we naturally got rid of as we iterated and testing was like a ball and chain that made everything super laborious.

TDD also represents a kind of first order thinking that assumes that if the individual parts are correct, the whole will likely be correct. It’s not wrong but it’s also very expensive to achieve. Software does have higher order effects.

It’s like the old car analogy. American car makers used to believe that if you QC every part and make unit tolerances tight, you’ll get a good car on final assembly (unit tests). This is true if you can get it right all the time but it made US car manufacturing very expensive because it required perfection at every step.

Ironically Japanese carmakers eschewed this and allowed loose unit tolerances, but made sure the final build tolerance worked even when the individual unit tolerances had variation. They found this made manufacturing less expensive and still produced very high quality (arguably higher quality since the assembly was rigid where it had to be, and flexible where it had to be). This is craftsman thinking vs strict precision thinking.

This method is called “functional build” and Ford was the first US carmaker to adopt it. It eventually came to be adopted by all car makers.

https://www.gardnerweb.com/articles/building-better-vehicles...

I believe that this is a fundamental problem of testing in all distributed systems: you are trying to test and validate for emergent behaviour. The other term we have for such systems is: chaotic. Good luck with that.

In fact, I have begun to suspect that the way we even think about software testing is backwards. Instead of test scenarios we should be thinking in failure scenarios - and try to subject our software to as much of those as possible. Define the bounding box of the failure universe, and allow computer to generate the testing scenarios within. EXPECT that all software within will eventually fail, but as long as it survives beyond set thresholds, it gets a green light.

In a way... we'd need something like a bastard hybrid of fuzzing, chaos testing, soak testing, SRE principles and probabilistic outcomes.

Emergent behaviour is complex, not chaotic. Chaos comes from sensitive dependence on initial conditions. Complexity is associated with non-ergodic statistics (i.e. sampling across time gives different results to sampling across space).

In fact it is not just wrong, but very wrong, as your auto example shows. Unfortunately engineers are not trained/socialised to think as holistically as perhaps they should be.

It's worth it to apply TDD to the pieces to which TDD is applicable. If not strict TDD than at least "test first" weak TDD.

The best candidates for TDD are libraries that implement pure data transformations with minimal integration with anything else.

(I suspect that the rabid TDD advocates mostly work in areas where the majority of the code is like that. CRUD work with predictable control and data flows.)

Also sometimes even if the individual parts aren’t right, the whole can still work.

Consider a function that handles all cases except for one that is rare, and testing for that case is expensive.

The overall system however can be written to provide mitigations upon composing — eg each individual function does a sanity check on its inputs. The individual function itself might be wrong (incomplete) but in the larger system, it is inconsequential.

Test effort is not a 1:1. Sometimes the test can be many times as complicated to write and maintain as the function being tested because it has to generate all the corner cases (and has to regenerate them if anything changes upstream). If you’re testing a function in the middle of a very complex data pipeline, you have regenerate all the artifacts upstream.

Whereas sometimes an untested function can be written in such a way where it is inherently correct from first principles. An extreme analogy would be the Collatz conjecture. If you start by first writing the test, you’d be writing an almost infinite corpus of tests — on the flip side, writing the Collatz function is extremely simple and correct up to large finite number.

Computer code is an inherently brittle thing, and the smallest errors tend to cascade into system crashes. Showstopper bugs are generated from off-by-one errors, incorrect operation around minimum and maximum values, a missing semicolon or comma, etc.

And doing sanity check on function inputs addresses only a small proportion of bugs.

I don't know what kind of programming you do, but the idea that a wrong function becomes inconsequential in a larger system... I feel like that just never happens unless the function was redundant and unnecessary in the first place. A wrong function brings down the larger system feels like the only kind of programming I've ever seen.

Physical unit tolerances don't seem like a useful analogy in programming at all. At best, maybe in sysops regarding provisioning, caches, API limits, etc. But not for code.

I think we’re talking extremes here. An egregiously wrong function can bring down a system if it’s wrong in just the right ways and it’s a critical dependency.

But if you look at most code bases, many have untested corner cases (which they’re likely not handling) but the code base keeps chugging along.

Many codebases are probably doing something wrong today (hence GitHub issues). But to catastrophize that seems hyperbolic to me. Most software with mistakes still work. Many GitHub issues aren’t resolved but the program still runs. Good designs have redundancy and resilience.

I can tell you that most (customer) issues in the software I work on are systemic issues, the database fails (widely used OSS) can corrupt under certain scenarios. They can be races, behaviour under failure modes, lack of correctness on some higher order (e.g. having half failed operations), the system not implementing the intent of the user. I would say very rarely those are issues that would have been caught by unit testing. Now integration testing and stress testing will uncover a lot of those. This is a large scale distributed system.

Now sometimes after the fact a unit test can somehow be created to reproduce the specific failure, possibly at great effort. That's not really something that useful at this point. You wouldn't write that in advance for every possible failure scenario (infinite).

All that said, sometimes there's attacks on systems that relate to some corner cases errors, which is a problem. Static analysis and fuzzers are IMO more useful tools in this realm as well. Also I think I'm hearing "dynamic/interepreted" language there (missing semicolons???). Those might need more unit testing to make up for the lack of compiler checks/warnings/type safety for sure.

The other point that's often missed is the drag that "bad" tests add to a project. Since it's so hard to write good tests when you mandate testing you end up with a pile of garbage that makes it harder to make progress. Other factors are the additional hit you take maintaining your tests.

Basically choosing the right kind of tests, at the right level, is judgement. You use the right tool for the right job. I rarely use TDD but I have used it in cases where the problem can relatively easily be stated in terms of tests and it helps me get quick feedback on my code.

EDIT: Also as another extreme thought ;) some software out there could be working because some function isn't behaving as expected. There's lots of C code out there that uses things that are technically UB but do actually have some guarantee under some precise circumstances (but idea but what can you do). In this case the unit test would pass despite the code being incorrect.

If anything I'm wondering the same question about you. If you find it so inconceivable that a bug is hiding in working code that is held up because the calling environment around it, than you must not have worked with big or even moderately sized codebases at all.

And in fact, fault tolerance with the assumption that all of it's parts are unreliable and will fail quickly makes for more fault tolerant systems.

The _processes and attitude_ that cause many individual parts to be incorrect will also cause the overall system to be crap. There's a definite correlation, but that correlation isn't about any specific part.

Yes it can, but the foundation is shaky, and having to make changes to it will tend to be scary.

Code is ultimately organic, and experienced developers know where the code needs be 100% and where the code can flex if needed. People have this idea that code is like mathematics where if one part fails, every part fails. To me if that is so, the design too tight and brittle and will not ship on time. But well designed code is more like an organism that has resilience to variation.

this time is more than made up for by the usual subsequent loss of debugging, refactoring and maintenance time, in my experience, at least for anything actively being used and updated

With TDD, the inner programming loop is:

1. form a belief about requirements

2. write a test to express that belief

3. write code to make that test pass

Without TDD, the loop is:

1. form a belief about requirements

2. write code to express that belief

3. futz around with manual testing, REPLs, and after-the-fact testing until you're sufficiently happy that the code actually does express that belief

And in my experience, the former loop is faster at producing working code.

  form a belief about a requirement
  write a test
  test fails
  write code
  test fails
  add debug info to code
  test fails no debug showing
  call code directly and see debug code
  change assert
  test fails
  rewrite test
  test succeed
  output test class data.. false 
  positive checking null equals null
  rewrite test
  test passes
  forget original purpose and stare at green passing tests with pride.

On a more serious note: just learn to use a debugger, and add asserts, if need be. To me TDD only helps having something that would run your code - but that's pretty much it. If you have other test harness options, I fail to see the benefits outside conference talks and books authoring.

This implies that the strength of the tests appears when it's modified?

Like the article says, TDD doesn't own the concept of testing. You can write good tests without submitting yourself to a dogma of red/green, minimum-passing (local-maximum-seeking) code. Debating TDD is tough because it gets bogged down with having to explain how you're not a troglodyte who writes buggy untested code.

And - on a snarkier note - this is a better argument against dynamic typing than for TDD.

TDD is actually quite good at manufacturing methods to reasonable tolerance, which the Japanese did require.

Higher level tests ensure the functional output is correct and typically don’t have built in any reliance on unit tests.

The times I have believed this myself, often turned out to be wrong when the full cost of development was taken into account. And I came back to the code later wishing I had tests around it. So you end up TDDing only the bug fix and exercising that part of the code with the failing test and then the code correction.

That was the time it took to actually write working code for that feature.

The version of "working code" that took 50% as long was just a con to fool people into thinking you'd finished until they move onto other things and a "perfectly acceptable" regression is discovered.

Say, initially you were told "if I click this button the status should update to complete", you write the test, you implement the code, rinse and repeat until a demo. During the demo, you discover that actually they'd rather the button become a slider, and it shouldn't say Complete when it's pressed, it should show a percent as you pull it more and more. Now, all the extra care you did to make sure the initial implementation was correct turns out to be useless. It would have been better to have spent half the time on a buggy version of the initial feature, and found out sooner that you need to fundamentally change the code by showing your clients what it looks like.

Of course, if the feature doesn't turn out to be wrong, then TDD was great - not only is your code working, you probably even finished faster than if you had started with a first pass + bug fixing later.

But I agree with the GP: unclear and changing requirements + TDD is a recipe for wasted time polishing throw-away code.

Edit: the second problem is well addressed by a sibling comment, related to complex interactions.

  > Say, initially you were told "if I click this button the status should  
  > update to complete", you write the test, you implement the code, rinse and 
  > repeat until a demo. During the demo, you discover that actually they'd 
  > rather the button become a slider, and it shouldn't say Complete when it's 
  > pressed, it should show a percent as you pull it more and more. Now, all the 
  > extra care you did to make sure the initial implementation was correct turns 
  > out to be useless.

This is an issue that's solved by something like Agile (frequent and regular stakeholder review, short cycle time) and has little to do with whether or not you've written tests first and let them guide your implementation; wrote the tests after the implementation was finished; or just simply chucked automated testing in the trash.

Either way, you've gotta make some unexpected changes. For me, I've really liked having the tests guide my implementation. Using your example, I may need to have a "percent complete" concept, which I'll only implement when a test fails because I don't have it, and I'll implement it by doing the simplest thing to get it to pass. If I approach it directly and hack something together I run the risk of overcomplicating the implementation based on what I imagine I'll need.

I don't have an opinion on how anyone else approaches writing complex systems, but I know what's worked for me and what hasn't.

Writing tests as you write the code is just regular and proper software development.

The former is more likely to adapt to the learning inherent in the writing of code, which someone above mentioned was easy to lose in TDD :)

1. Write test, see that it fails the way you expect.

2. Write code that makes the test pass.

3. Write test...

and be secure that you can fearlessly refactor and not backslide while you play with different ideas so long as all your tests stay green.

I would get overwhelmed so fast if I just had 50 failing tests and no implementation.

One of the above comments mentions BDD as a close cousin to TDD, but that is wrong as TDD is actually BDD as you should only be testing behaviours, which allow you to "fearlessly refactor"

https://softwareengineering.stackexchange.com/questions/1236...

Which seems like the exact opposite of what the talk is saying you should do.

Here’s how I would do it. The challenge is that the result can’t be precisely defined because it’s essentially art. But with TDD the assertions don’t actually have to actually live in code. All we have to do is make incremental verifiable progress that lets us fearlessly make changes.

So I would set up my viewport as a grid where in each square there will eventually live a rendered image or animation. The first one blank, the second one a dot, the third a square, the fourth with color, the fifth a rhombus, the sixth with two disjoint rhombuses …

When you’re satisfied with each box you copy/paste the code into the next one and work on the next test always rendering the previous frames. So you can always reference all the previous working states and just start over if needed.

So the TDD flow becomes

1. Write down what you want the result of the next box to look like.

2. Start with the previous iteration and make changes until it looks like what you wanted.

3. Write down what you want…

(Note: manual verification is not the same as manual execution!)

You think of a small chunk of functionality you are comfident about, write the tests for that (some people say just one test, i am happy with up to three or so), then write the implementation that makes those tests pass. Then you refactor. Then you pick off another chunk and 20 GOTO 10.

If at some point it turns out your belief about the functionality was wrong, fine. Delete the tests for that bit, delete the code for it, make sure no other tests are broken, refactor, and 20 GOTO 10 again.

The process of TDD is precisely about writing code when you don't know how the program is going to work upfront!

On the other hand, implementing a well-defined spec is when TDD is much less useful, because you have a rigid structure to work to in both implementation and testing.

I think the biggest problem with TDD is that completely mistaken ideas about it are so widespread that comments like this get upvoted to the top even on HN.

Is general understanding of TDD really that far off the mark? I had no idea, and I've been doing this for essentially 2 decades now.

Lets say my designer came to me and now wants bump mapped textures on engine, well I cannot touch that compute shader without writing a test first, so suggestions for a TDD framework for GPU shaders.

Once frozen the test compares the picture (or other artefact) against the generated picture (or artefact).

Im not sure if it'd be useful in your situation though.

The code changes that changed the UI - even changing some CSS - would cause a screenshot comparison failure on certain steps in the test. If it is what was expected then we overwrote the old screenshot with a new one.

It isnt exactly the same as the TDD process coz sometimes you write the code first (e.g. CSS), eyeball it and then rebuild the screenshot if it looks correct.

I'd say it's close enough though.

I wont pretend it worked perfectly. The screenshot comparison algorithms were sometimes flaky and UIs can be surprisingly non-determinstic and you need to have a process to pull the database and update screenshots accordingly. However, it's the approach I'd prefer to take in the future (I havent done UIs for about 3 years now).

I also wasnt religious about covering every single scenario with tests but I could have been. The company moved fast and sometimes they just wanted quick and not reliable.

Or perhaps you've worked with some real TDD zealots, that doesn't sound like fun.

The folks I've worked with use these as guiding recommendations, not binding dictates.

For some of the UI stuff you mentioned elsewhere, I've seen a stronger focus on testing not just business logic, but view logic as well (where possible), but generally not to the degree of testing the rendered output of the UI. Maybe that's a thing somewhere, but I haven't personally seen it.

But yeah, if you dont have one of those tools or its super unreliable or it's only available in a language you cant use then you cant do this.

I dont really consider this to be bending the rules of TDD. It's more like next gen TDD IMO.

If you're thinking of unit tests as the thing that catches bugs before going to production and proves your code is correct, and want to write a suite of tests before writing code, that is far beyond the capabilities of most software engineers in most orgs, including my own. Some folks can do it, good for them.

But if you think of unit tests as a way to make sure individual little bits of your code work as you're writing them (that is, you're testing "the screws" and "the legs" of the tables, not the whole table), then it's quite simple and really does save time, and you certainly do not need full specs or even know what you're doing.

Write 2-3 simple tests, write a function, write a few more tests, write another function, realize the first function was wrong, replace the tests, write the next function.

You need to test your code anyway and type systems only catch so much, so even if you're the most agile place ever and have no idea how the code will work, that approach will work fine.

If you do it right, the tests are trivial to write and are very short and disposable (so you don't feel bad when you have to delete them in the next refactor).

Do you have a useful test suite to do regression testing at the end? Absolutely not! In the analogy, if you have tests for a screw attaching the leg of a table, and you change the type of legs and the screws to hook them up, of course the tests won't work anymore. What you have is a set of disposable but useful specs for every piece of the code though.

You'll still need to write tests to handle regressions and integration, but that's okay.

But the end result of writing tests is often that you create a lot of testing tied to what should be implementation details of the code.

E.g. to write "more testable" code, some people advocate making very small functions. But the public API doesn't change. So if you test only the internal functions, you're just making it harder to refractor.

This is the major issue I have with blind obedience to TDD.

It often feels like the question of "What SHOULD this be doing" isn't asked and instead what you end up with is a test suite that answers the question "What is this currently doing?"

If refactoring code causes you to refactor tests, then your tests are too tightly coupled to implementation.

Perhaps the missing step to TDD is deleting or refactoring the test at the end of the process so you better capture intent rather than the flow of consciousness.

Example: I've seen code that had different code paths to send in a test "logger" to ensure the logger was called at the right locations and said the right messages. That made it difficult add new information to the logger or add new logger messages. And for what?

You're not supposed to write every single test upfront, you write a tiny test first. Then you add more and refactor your code, repeat until there is nothing left of that large complicated thing you were working on.

There are also people who test stupid things and 3rd party code in their tests and either they get a fatigue from it and/or think their tests are well written.

The raison d'etre of TDD is that developers can't be trusted to write tests that pass for the right reason - that they can't be trusted to write code that isn't buggy. Yet it depends on them being able to write tests with enough velocity that they're cheap enough to dispose?

But yeah, trying to write all the tests for a whole big component up front, unless it's for something with a stable spec (eg. I once implemented some portions of the websockets spec in servo, and it was awesome to have an executable spec as the tests), is usually an exercise in frustration.

People blame engineers for not writing tests or doing TDD when, if they did, they would likely be replaced with someone who can churn out code faster. It is rare, IME, to have culture where the measured and slow progress of TDD is an acceptable trade off.

As always, the job of a paid software person is to merge what the product people want with what good software quality requires (and what power a future version will unleash). Implement valuable things in software in a way that makes the future of that software better and more powerful.

I would however be more amenable to e.g. Prototyping first, and then using that as a guide for TDD. Not sure if there is a name for that approach though. "spike" maybe?

[1] https://www.machow.ski/posts/galls-law-and-prototype-driven-...

Interesting - for me, that's the only time I truly practice TDD, when I don't know how the code is going to work. It allows me to start with describing the ideal use case - call the API / function I would like to have, describe the response I would expect. Then work on making those expectations a reality. Add more examples. When I run into a non-trivial function deeper down, repeat - write the ideal interface to call, describe the expected response, make it happen.

For me, TDD is the software definition process itself. And if you start with the ideal interface, chances are you will end up with something above average, instead of whatever happened to fall in place while arranging code blocks.

So the TDD OP describes here is not an Agile TDD.

The normal TDD process is:

    1. add one test
    2. make it (and all others) pass
    3. maybe refactor so code is sane
    4. back to 1, unless you're done.

I 100% agree with capturing requirements in tests. However, I argue that TDD does not cause that to happen.

I'd even make a stronger statement. Automated tests that don't capture a requirement should be deleted. Those sorts of tests only serve to hinder future refactoring.

A good test for a sort method is one that verifies data is sorted at the end of it. A bad test for a sort method is one that checks to see what order elements are visited in the sorting process. I have seen a lot of the "element order visit" style tests but not a whole lot of "did this method sort the data" style tests.

I don't see the connection between TDD and your bad tests examples though.

I would test a sort method just the way you describe, using TDD.

So, you say "Ok, I'll need a `heapify` method and a `siftDown` methods, so I'm going to write the tests to make sure both of those are working properly". But remember, we started this discussion saying "we need a sort method". So now, if you decide "You know what, heapsort is garbage, let's do tim sort instead!" Now, all the sudden you've got a bunch of useless tests. In the best case, you can simply delete those tests, but often devs often get intimidated deleting such tests "What if something else needs the `heapify` method"?

And that's exactly the problem I was pointing out with the example. We started the convo saying "tests couple implementation" and that's what's happened here. Our tests are making it seem like heap sort is the implementation we should use when we started this convo, we just needed a sorting method.

But now imagine we are talking about something way more complicated and/or less well known than a sorting algorithm. Now, it becomes a lot harder to sift out which tests are for implementation things and which are for requirements things. Without deleting the "these tests make sure I did a good implementation" future maintainers of the code are left to guess on what's a requirement and what's an implementation detail.

All sort methods can and should be tested with the same tests, that assert that unsorted input is converted to sorted output.

Choosing heapsort for performance reasons should normally not affect your test suite at all.

That's not to say you can't seek improvement. The public interface can be expanded without impacting existing uses. If, for example, an existing function doesn't reflect your current view of the world add a new one rather than try to jerry-rig the old one. If the new solutions are radically different such that you are essentially rewriting your code from scratch, a clean break is probably the better route to go.

If you are confident that existing users are no longer touching your legacy code, remove it rather than refactor it.

Things change. You change the code in response. What broke? Without the tests, you don't know.

"Things change" include "you fixed a bug". Bug fixes can create new bugs (the only study I am familiar with says 20-50% probability). Did your bug fix break anything else? How do you know? With good test coverage, you just run the tests. (Yes, the tests are never complete enough. They can be complete enough that they give fairly high confidence, and they can be complete enough to point out a surprising number of bugs.)

Does that make you pay "at least twice"? No. It makes you pay, yes, but you get a large amount of value back in terms of actually working code.

You aren't doing "double the work" even though it seems that way on paper, unless the problem was solved with brittle architectural foundations and tightly coupled tests.

At the heart of this problem is most developers don't quite grasp boundary separation intuitively I think.

Even if you know exactly how the software is going to work, how would you know if your test cases are written correctly without having the software to run them against? For that reason alone, the whole idea of TDD doesn't even make sense to me.

Have been meaning to try mutation testing as a way of sussing out tests that cover lines but don't actually test behavior.

The reason is, the tests and the code are symbiotic, your tests prove the code works and your code proves the tests are correct. TDD guarantees you always have both of those parts. But granted it is not the only way to get those 2 parts.

You can still throw into the mix times when a bug is present, and it is this symbiotic relationship that helps you find the bug fast, change the test to exercise the newly discovered desired behaviour, see the test go red for the correct reason and then make the code tweak to pass the test (and see all the other tests still pass).

What I find is much much better approach is what I call "detached test development" (DTD). The idea is: 2 separate teams get the requirements; one team writes code, the other write tests. They do not talk to each other! Fist when a test is not passed, they have to discuss: is the requirement not clear enough? What is the part that A thought about, but not B? Assignment of tests and code can be mixed, so a team makes code for requirements 1 through 100, and tests for 101 to 200, or something like that. I had very very good results with such approach.

TDD is a feedback cycle, you write small increments of tests before writing a small bit of a code. You don't write a bunch of tests upfront, that'd be silly. The whole point is to integrate small amounts of learning as you go, which help guide the follow-on tests, as well as the actual implementation, not to mention questions to need to ask the broader business.

Your DTD idea has been tried a lot in prior decades. In fact, as a student I was on one of those testing teams. It's a terrible idea, throwing code over a wall like that is a great way to radically increase the latency of communication, and to have a raft of things get missed.

I have no idea why there's such common misconceptions of what TDD is. Maybe folks are being taught some really bad ideas here?

100%. Metrics of quality are really really hard to define in a way that are both productive and not gamified by engineers.

> What I find is much much better approach is what I call "detached test development" (DTD)

I'm a test engineer and some companies do 'embed' an SDET like the way you mention within a team - it's not quite that clear cut, they can discuss, but it's still one person implementing and another testing.

I'm always happy to see people with thoughts on testing as a core part of good engineering rather than an afterthought/annoyance :)

This feels a bit like when you write a layer of encapsulation to try to make a problem easier only to discover that all of the complexity is now in the interface. Isn't converting the PO's requirements into good, testable requirements the hard technical bit?

Therefore, TDD's secret sauce is in concretely forcing developers to think through requirements, mental models etc. and quantify them in some way. When you hit a block, you need to ask yourself whats missing, then figure out, and continue onward, making adjustments along the way.

This is quite malleable to unknown unknowns etc.

I think the problem is most people just aren't chunking down the steps of creating a solution enough. I'd argue that the core way of approaching TDD fights most human behavioral traits. It forces a sort of abstract level of reasoning about something that lets you break things down into reasonable chunks.

TDD is not a silver bullet, it's one tool among many.

So much of this is because TDD has become synonymous with unit testing, and specifically solitary unit testing of minimally sized units, even though that was often not the original intent of the ideators of unit testing. These tests are tightly coupled to your unit decomposition. Not the unit implementation (unless they're just bad UTs), but the decomposition of the software into which units/interfaces. Then the decomposition becomes very hard to change because the tests are exactly coupled to them.

If you take a higher view of unit testing, such as what is suggested by Martin Fowler, a lot of these problems go away. Tests can be medium level and that's fine. You don't waste a bunch of time building mocks for abstractions you ultimately don't need. Decompositions are easier to change. Tests may be more flaky, but you can always improve that later once you've understood your requirements better. Tests are quicker to write, and they're more easily aligned with actual user requirements rather than made up unit boundaries. When those requirements change, it's obvious which tests are now useless. Since tests are decoupled from the lowest level implementation details, it's cheap to evolve those details to optimize implementation details when your performance needs change.

This is a trouble I often see expressed about static types. And it’s an intuition I shared before embracing both. Thing is, embracing both helped me overcome the trouble in most cases.

- If I have a type interface, there I have the shape of the definition up front. It’s already beginning to help verify the approach that’ll form within that shape.

- Each time I write a failing test, there I have begun to define the expected behavior. Combined with types, this also helps verify that the interface is appropriate, as the article discusses, though not in terms of types. My point is that it’s also verifying the initial definition.

Combined, types and tests are (at least a substantial part of) the definition. Writing them up front is an act of defining the software up front.

I’m not saying this works for everyone or for every use case. I find it works well for me in the majority of cases, and that the exception tends to be when integrating with systems I don’t fully understand and which subset of their APIs are appropriate for my solution. Even so writing tests (and even sometimes types for those systems, though this is mostly a thing in gradually typed languages) often helps lead me to that clarity. Again, it helps me define up front.

All of this, for what it’s worth, is why I also find the semantics of BDD helpful: they’re explicit about tests being a spec.

This is true, and I think that's why TDD is a valuable exercise to disambiguate requirements.

You don't need to take an all/nothing approach. Even if you clarify 15-20% of the requirements enough to write tests before code, that's a great place to begin iterating on the murky 80%.

Because for years people have practice with defining software iteratively, whether by choice or being forced by deadlines and agile.

That doesn't inherently make one or the other harder, it's just another familiarity problem.

TDD goes nicely with top-down design using something like Haskell's undefined to stub out functionality that typechecks and it's where clauses.

    myFunction = haveAParty . worldPeace . fixPoverty $ world
        where worldPeace = undefined
                   haveAParty = undefined
                   fixPoverty = undefined

By the time I've reached a point where the feature can actually be tested, I end up with a pretty good skeleton of what tests should be written.

There's a hidden benefit to doing this, actually. It frees up your brain from keeping that running tally of "the feature should do X" and "the feature should guard against Y", etc. (the very items that go poof when you get distracted, mind you)

Try it. Write a test for 1, and an implementation which passes that test then for 2, and so on.

Bellow is something written without any TDD (in Java)

    private static String convert(int digit, String one, String half, String ten)     {
    switch(digit) {
    case 0: return "";
    case 1: return one;
    case 2: return one + one;
    case 3: return one + one + one;
    case 4: return one + half;
    case 5: return half;
    case 6: return half + one;
    case 7: return half + one + one;
    case 8: return half + one + one + one;
    case 9: return one + ten;
    default:
    throw new IllegalArgumentException("Digit out of range 0-9: " + digit);
    }
    }

    public static String convert(int n) {
    if (n > 3000) {
    throw new IllegalArgumentException("Number out of range 0-3000: " + n);
    }

    return convert(n / 1000, "M", "", "") + 
        convert((n / 100) % 10, "C", "D", "M") +
        convert((n / 10) % 10, "X", "L", "C") +
                convert(n % 10, "I", "V", "X");
}

If you have "a growing collection of broken tests", that's not TDD. That's "they told us we have to have tests, so we wrote some, but we don't actually want them enough to maintain them, so instead we ignore them".

Tests help massively with iterating a design on a partly-implemented code base. I start with the existing tests running. I iterate by changing some parts. Did that break anything else? How do I know? Well, I run the tests. Oh, those four tests broke. That one is no longer relevant; I delete it. That other one is testing behavior that changed; I fix it for the new reality. Those other two... why are they breaking? Those are showing me unintended consequences of my change. I think very carefully about what they're showing me, and decide if I want the code to do that. If yes, I fix the test; if not, I fix the code. At the end, I've got working tests again, and I've got a solid basis for believing that the code does what I think it does.

> The obvious exception to this, where I still use TDD, is when implementing a well defined spec.

From my understanding (and experience), TDD is quite the opposite. It's most useful when you don't have the spec, don't have clue how software will work in the end. TDD creates the spec, iteratively.

    1. Hack in what I want in some exploratory way
    2. Write good tests
    3. Delete my hacks from step 1, and ensure all my new tests now fail
    4. Re-implement what I hacked together in step 1
    5. Ensure all tests pass

Once we established that that works properly, we think of a way to do it clean and tested for all other usecases.

(In the book "the pragmatic programmer" it's called "tracer bullets / code".)

Since typing speed was never the bottleneck in software development. Throwing away the little bit of code you wrote, it not expensive. And writing it back in with TDD is incredibly efficient.

We typically write acceptance tests, and they have been helpful either early on or later in our product development lifecycle.

Even if software isn't defined upfront, the end goal is likely defined upfront, isn't it? "User X should be able to get data about a car," or "User Y should be able to add a star ratings to this review," etc.

If you're building a product where you're regularly throwing out large parts of the UI / functionality, though, I suppose it could be bad. But as a small startup, we have almost never been in that situation over the many years we've been in business.

In many ways I guess I lean maximalist in my practices, and find it helpful, but I'd readily concede that the maximalist advocates are annoying and off-putting. I once had the opportunity to program with Ward Cunningham for a weekend, and it was a completely easygoing and pragmatic experience.

But spikes are written to be thrown away. You never put them into production. Production code is always written against some preexisting test, otherwise it is by definition broken.

I'm not sure what you mean by this. Why are the tests you're writing not "adequate" for the code you're testing?

If I read into this that you're using code coverage as a metric -- and perhaps even striving for as close to 100% as possible -- I'd argue that's not useful. Code coverage, as a goal, is perhaps even harmful. You can have 100% code coverage and still miss important scenarios -- this means the software can still be wrong, despite the huge effort put into getting 100% coverage and having all tests both correct and passing.

Functionality based on a set of initial specs and a hazy understanding of the actual problem you are trying to solve might on the other hand might not be worth investing in protecting.

It might work if you are starting on some new, relatively self-contained feature...

If you can’t do that yet, you spike, mess around with it, get your bearings and some insight into where you want to go, try a few things out. Then, trash that and then write the first test as above.

It almost always works, and excels somewhat in legacy code vs just changing things.

It isn’t about ‘might work’. The TDD paradigm isn’t about finding shiny places where you can use. It is just a nice way of writing clean maintainable software in almost all areas.

Places where TDD won’t work are usually by exception rather than the norm.

Often those exceptions are where something already exists and wasn’t put in by TDD because by the definition of TDD you wouldn’t have been able to write that in the first place.

And that is, write code, chuck it away, start again.

Prototype your feature without TDD. Then chuck it away and build it again with TDD.

My guess is by doing so code quality and reduced technical debt pay more than what is lost in time.

Very few companies work like this I imagine: None that I have worked for.

Since keyboard typing is a short part of software development it is probably a great use of time and could catch more bugs and design quirks early on when they cost $200/h instead of $2000/h.

Another partly orthogonal issue is that design is important for some problems, and you don't usually reach a good design by chipping away at a problem in tiny pieces.

TDD fanatics insist that it works for everything. Do I believe them that it improved the quality of their code? Absolutely; I've seen tons of crap code that would have benefited from any improvement to the design, and forcing it to be testable is one way to coerce better design decisions.

But it really only forces the first-order design at the lowest level to be decent. It doesn't help at all, or at least not much, with the data architecture or the overall data flow through the application.

And sometimes the only sane way to achieve a solid result is to sit down and design a clean architecture for the problem you're trying to solve.

I'm thinking of one solution I came up with for a problem that really wasn't amenable to the "write one test and get a positive result" approach of TDD. I built up a full tree data structure that was linked horizontally to "past" trees in the same hierarchy (each node was linked to its historical equivalent node). This data structure was really, really needed to handle the complex data constraints the client was requesting. As yes, we pushed the client to try to simplify those constraints, but they insisted.

The absolute spaghetti mess that would have resulted from TDD wouldn't have been possible to refactor into what I came up with. There's just no evolutionary path between points A and B. And after it was implemented and it functioned correctly--they changed the constraints. About a hundred times. I'm not even exaggerating.

Each new constraint required about 15 minutes of tweaking to the structure I'd created. And yes, I piled on tests to ensure it was working correctly--but the tests were all after the fact, and they weren't micro-unit tests but more of a broad system test that covered far more functionality than you'd normally put in a unit test. Some of the tests even needed to be serialized so that earlier tests could set up complex data and states for the later tests to exercise, which I understand is also a huge No No in TDD, but short of creating 10x as much testing code, much of it being completely redundant, I didn't really have a choice.

So your point about the design changing as you go is important, but sometimes even the initial design is complex enough that you don't want to just sit down and start coding without thinking about how the whole design should work. And no methodology will magically grant good design sense; that's just something that needs to be learned. There Is No Silver Bullet, after all.

True, but… you can still design the architecture, outlining the solution for the entire problem, and then apply TDD. In this case your architectural solution will be an input for low level design created in TDD.

I described a situation where TDD really, really, really wouldn't have worked. The whole structure needed to be developed, or at least 80% of it, before it would have made sense to write any tests--and the actual TDD philosophy would be to write "one small test" and only write exactly as much code as required to satisfy the test.

The sane approach was to create the entire structure based on the design, and then test it after it was complete as an entire system. Some of the micro-functionality that TDD would have had you test would have become technical debt as a change-detector later when the client changed their specific requirements.

As I said above, there is no evolutionary path from tiny pieces to the full structure, and TDD requires that you follow such an evolutionary path. If you're writing a bunch of tests and then creating a nontrivial amount of code, then you're following test-first, but not really following TDD. And I question even how valuable that is when you don't necessarily understand what would need to be tested before you've finished implementing the system.

Regulations are always ambiguous, standards are never followed, and widely implemented standards are never implemented the way the document tells.

You will probably still gain productivity by following TDD for those, but your process must not penalize too much changes in spec, because it doesn't matter if it's written in Law, what you read is not exactly what you will create.

- The Tao of Programming (1987)

I created dozens of “edge case” sample spreadsheets with horrible things in them like Bad Strings in every property and field. Think control characters in the tab names, RTL Unicode in the file description, etc…

I found several bugs… in Excel.

As a made up example. The "what" of the program is to take in a bunch of transactions and emit daily summaries. That's a straight forward "what". It however leaves tons of questions unanswered. Where does the data come from and in what format? Is it ASCII or Unicode? Do we control the source or is it from a third party? How do we want to emit the summaries? Printed to a text console? Saved to an Excel spreadsheet? What version of Excel? Serialized to XML or JSON? Do we have a spec for that serialized form? What precision do we need to calculate vs what we emit?

So the real "what" is: take in transaction data encoded as UTF-8 from a third party provider which lives in log files on the file system without inline metadata then translate the weird date format with only minute precision and lacking an explicit time zone and summarize daily stats to four decimal places but round to two decimal places for reporting and emit the summaries as JSON with dates as ISO ordinal dates and values at two decimal places saved to an FTP server we don't control.

While waiting for all that necessarily but often elided detail you can either start writing some code with unit test or wait and do no work until you get a fully fleshed out spec that can serve as the basis for writing tests. Most organizations want to start work even while the final specs of the work are being worked on.

Is that significant? Your tests can start to answer these unanswered questions before you ever get around to writing implementation. Suppose you thought you wanted to write data in ASCII format. But then you write some test cases and realize that you actually need Unicode symbols. Now you know what your implementation needs to do.

Testing is the spec. The exact purpose of testing, which in fairness doesn't have the greatest name, is to provide documentation around what the program does. That it is self-verifying is merely a nice side effect. There is no need for all the questions to be answered while writing the spec (a.k.a. tests). You learn about the answers as you write the documentation. The implementation then naturally follows.

If your customer decides they want/need ASCII your test is immaterial. The same is true for writing any tests before you've got a meaningful specification. You're just writing code to write code. At that stage it makes more sense to write a scaffold in the general shape of the task than a bunch of tests defining precise but made up requirements.

Aspirational tests are great if you know exactly what you need to do. They tell you when you get there. If you don't know exactly what you need to do they're just wasted effort that get thrown away with nothing having been gained.

Not at all. First, if your customer doesn't know upfront that ASCII is necessary, showing the customer how the program will work will help them realize that. The lesson gained from your test was useful in getting to get to that point.

Second, your failed attempt at an interface that provides UTF-8 (or whatever) during exploration is convertible to a negative case that will fail if ASCII isn't used, documenting the customer's requirement for future developers.

If the customer originally required UTF-8 and years later a changing landscape forced them to require ASCII instead, again your spec can be negated to help you ensure that you made the correct modifications to meet the new requirements.

> Aspirational tests are great if you know exactly what you need to do.

Whereas I would argue that TDD isn't all that useful if you know exactly what you need to do. In that case you can simple write the code along with unit/integration/acceptance tests that validate behaviour.

TDD is about exploring the unknowns and answering questions – showing the customer how the program will function on the outside – before you waste time implementing all the internal details of a full program around UTF-8 only to be told when you demo it that the customer actually requires ASCII, potentially requiring massive rework. The latter is where you waste effort.

Often times you write to find what you want to accomplish. It sounds backwards, perhaps it is backwards, but it's also very human. Without something to show the user, they often have no idea what they want. In fact, people are far better at telling you what's wrong with what's presented to them then enumerating everything they want ahead of time.

TDD is great but also completely useless for sussing requirements out of users.

If you know the exact requirements upfront, you don't really need TDD. You can simply write the code to those requirements and, perhaps, add in some automated acceptance testing to help catch mistakes. TDD shines when you are unsure and exploring options.

Maybe we need to learn how to delete stuff that doesn't make sense.

Get rid of broken test. Get rid of incorrect documentation.

Don't be afraid to delete stuff to improve the overall program.

The result was that 1) the devs were exceptionally happy, 2) the TL was mostly happy, except with some of the extra forced work he created for himself as the bottleneck, 3) the project took longer than expected, and 4) the code was SOOOOO readable but also very inefficient. We realized during the project that forcing unit tests for literally everything was also forcing a breaking up of methods & functions into much smaller discrete pieces than would have been optimal from both performance & extensibility perspectives.

It wasn't the last TDD project we ran, but we were far more flexible after that.

I had one other "science project" while managing that team, too. It was one where we decided to create an architect role (it was the hotness at that time), and let them design everything from the beginning, after which the dev team would run with it using their typical agile/sprint methodology. We ended up with the most spaghetti code of abstraction upon abstraction, factories for all sorts of things, and a codebase that became almost unsupportable from the time it was launched, necessitating v2.0 be a near complete rewrite of the business logic and a lot of the data interfaces.

The lessons I learned from those projects was that it's important to have experienced folks on every dev team, and that creating a general standard that allows for flexibility in specific architectural/technical decisions will result in higher quality software, faster, than if one is too prescriptive (either in process or in architecture/design patterns). I also learned that there's no such thing as too much SQA, but that's for a different story.

I'm not quite sure what is right or wrong about my approach, but I do find the code tends to work and work reliably once it compiles and the tests pass.

Very few critics notice this.

I write tdd when doing advent of code. And it’s not that I set out to do it or to practice it or anything. It just comes very natural to small, well defined problems.

Nobody out there is writing all their tests up front.

TDD is an iterative process, RED GREEN REFACTOR.

- You write one test.

- Write JUST enough code to make it pass.

- Refactor while maintaining green.

- Write a new test.

- Repeat.

I don't want this to come off the wrong way but what you're describing shows you are severely misinformed about what TDD actually is or you're just making assumptions about something based on its name and nothing else.

Everyone understands the idea, it's just a massive time sink for no more benefit than a test-after methodology provides.

> Everyone understands the idea, it's just a massive time sink for no more benefit than a test-after methodology provides.

This is not something I agree with. In my experience, when TDD is used you come up with solutions to problems that are better than what you'd come up with otherwise and it generally takes much less time overall.

Writing tests after ensures your code is testable. Writing your tests first ensures you only have to write your code once to get it under test.

Again, you don't always need TDD and applying it when you don't need it will likely be a net time sink with little benefit.

> - Write JUST enough code to make it pass.

Those two steps aren't really trivial. Even just writing the single test might require making a lot of design decisions that you can't really make up-front without the code.

Sadly TDD is right up there with REST in being almost universally misunderstood.

That's a flaw in TDD and REST, not in the universe.

And if not, perhaps there was a preconceived idea of the code block and what it was going to do, rather than specifying the behaviour wanted via the RGR cycle. With a preconceived idea, with or without the tests, if that idea is wrong, you’ll hit the dead end and have to back track. Fortunately I find that even though I do sometimes find myself in this situation, quite often those tests can be repurposed fairly quickly rather than being chucked away, after all the tests are still software, and not hardware.

> - You write one test

Easier said than done. Say your task is to create a low level audio mixer which is something you've never done before. Where do you even begin? That's the hard part.

Some other commenters here have pointed out that exploratory code is different from TDD code, which is a much better argument then what you made here imo.

> I don't want this to come off the wrong way but what you're describing shows you are severely misinformed about what TDD actually is or you're just making assumptions about something based on its name and nothing else.

Instead of questioning the OP's qualifications, perhaps you should hold a slightly less dogmatic opinion. Perhaps OP is familiar with this style of development, and they've run into problem firsthand when they've tried to write tests for an unknown problem domain.

I find that iterating on tests in exploratory code makes for an excellent driver to exercise the exploration. I don’t see the conflict between the two, except I am not writing test cases to show correctness, I am writing them to learn. To play with the inputs and outputs quickly.

I've seen loads of examples where the tests haven't been updated in years to take account of new functionality. When that happens you aren't really doing TDD anymore.

The article actually discusses whether this is accurate or not. TDD started out as a testing process but got adopted for its design consequences which is why there is a lot of confusion.

Naming it test driven design would have gone a long way to help things and also resulted in less cargo culting. "Have to TDD all day or you don't do TDD"