voiceofunreason
voiceofunreason on Hacker News
I think this paper is best understood as an experience report: after reviewing [list of references], we thought this would be an interesting experiment to try, and here is a summary of our results and conclusions.
"Unlike Beck’s original three-phase cycle, AI-TDD does not assign Refactor to a dedicated persona."
If I ran the zoo, I'd have leaned toward the "many more much smaller steps" option, with at least two distinct persona for refactoring (and probably more than one distinct test writer as well).
But how much do you want to invest in a higher fidelity representation of "expert" TDD, given the lack of evidence that TDD is a significant improvement on YOLO?
- request-with-a-body
- idempotent-request-with-a-body
- safe-request-with-a-body
because the additional constraints induce properties that are extremely useful to general purpose clients ("I didn't get a reply to my idempotent-request-with-a-body, can I resend it without risking loss of property?")
Would someone then come along an introduce safe-request-without-a-body method? After all, we can already meet that "need" with safe-request-with-a-body and content-length: 0.
Think rfc-5789::PATCH - mechanically, it's just another request-with-a-body, but with more tightly constrained semantics. But general purpose components can take advantage of the additional properties, and so we introduce a "niche" method with tighter constraints.
Document resource manipulation is a common case, so we probably end up with a family of specialized methods, in much the same way that we have a bunch of WebDAV methods.
It does seem that way - this one theme does drown out all of the other criticisms.
That said, (to first order) everyone's been "doing it wrong" for 25 years (give or take, there are lots of reasonable start dates to choose from).
Maybe "everyone" isn't the actually the problem, and we need to be open to alternatives like "our theory of what makes it work isn't very strong yet", "we oversold it", or "it was only ground breaking when compared with just-winging-it".
The best "let's do something not trivial" TDD book is probably still _Growing Object Oriented Software, Guided by Tests_ by Steve Freeman and Nat Pryce.
Most of the "more than the basics" topics are the same kinds of practices that were considered "good design" whether you were using TDD or not. For example, Parnas 1971, Berard 1993, John Carmack 1998 ("Time is an input...."), and so on.
If you are interested in more than the basics on TDD, the right starting point is _Test Driven Development by Example_ by Kent Beck, which while a bit thin on examples actually covers a nice variety of more advanced topics (although not in great depth). If you are going this route, you should pair that with Beck's 2023 essay "Canon TDD".
Same idea, different spelling: do you really think TDD should get credit for your good results, when you aren't actually shackling yourself to the practices that the thought leaders in that community promote?
A small community of programmers, with a disproportionately large audience, foretold that practicing test-driven development would produce great benefits; over twenty five years the audience has found that not to be the case.
Compare with "continuous integration" - here, the immediate returns of trying the proposed discipline were so good that pretty much everybody who tried the experiment got positive returns, and leaned into it, and now CI (and later CD) are _everywhere_.
As for what is gained, try this spelling: test driven development adds load to your interfaces at a time when you know the least about the problem you are trying to solve, which is to say the period where having your interfaces be flexible is valuable.
And thus, the technique gets criticism from both ends -- that design work that should have been done up front is deferred (making the design more difficult to change, therefore introducing costs/delays), and that the investment is being made in testing before you have a clear understanding for which tests are going to be sensitive to the actual errors that you introduce creating the code (thereby both increasing the amount of "waste" in the test suite, in addition to increasing the risk of needing test rewrites).
The situation is further not improved by (a) the fact that most TDD demonstrations are problems that are small, stable problems that you can solve in about an hour with any technique at all and (b) the designs produced in support of the TDD practice aren't clearly an improvement on "just doing it", and in some notable cases have been much much worse.
So if it is working for you: GREAT, keep it up; no reason for you not to reap the benefits if your local conditions are such that TDD gives you the best positive return on your investment.
The short version is that "unit test" did actually mean something (see Glenford Myers, __The Art of Software Testing__ or Boris Beizer, __Software Testing Techniques__), although it wasn't necessarily clear how those definitions applied to object-oriented programming (see Robert Binder, __Testing Object-Oriented Systems__).
The Test-First/TDD/XP community later made an effort to pivot to the language of "programmer test", but by the time that effort began it was already too late.
So I think you should continue to call your tests "tests" (or "checks", if you prefer the framing of James Bach and Michael Bolton).
As best I can tell - there's no historicity to the idea that "unit test" was a reference to the isolation of a tests from its peers; it's just a ret-con.
No it isn't. Evidence: I'm reading this in a web browser.
"...REST is intended for long-lived network-based applications that span multiple organizations. If you don’t see a need for the constraints, then don’t use them."
Bikeshedding the spelling of resource identifiers? Or what "verb" should be used to express specialized domain semantics? Yeah, _that_ is certainly plague bullshit.
Is the label "TDD" being hijacked for something new? Did that already happen? Are LLMs now responsible for defining TDD?
https://web.archive.org/web/20071214085409/http://www.itmweb...
Beck's book was 2003, with examples in Java and Python. David Astels wrote a book later that year, again primarily Java but also with short chapters discussing RubyUnit, NUnit, CppUnit.... Growing Object Oriented Software was 2009.
My guess is that "peak" is somewhere between 2009 and 2014 (the DHH rant); after that point there aren't a lot of new voices with new things to say ("clap louder" does not qualify as a new thing to say).
That said, if you're aware of the gap between decision and feedback, and managing that gap, I don't think it matters very much whether that feedback comes in the form of measurements of runtime behavior vs analysis of the source text itself. It might even make sense to use a mixed strategy (preferring the dynamic measurements when the design is unstable, but switching strategies in areas where changes are less frequent).
Disclaimer first: TDD won't give you anything that you couldn't instead achieve via "git gud"; except perhaps a reduced anxiety about overlooking a subtle error (but after "git gud", you don't _make_ subtle errors, do you?)
The main justification for test first is something like this: "we didn't have to be brilliantly prescient designers to find a less tightly coupled design. We just had to be intolerant of pointless effort in writing the tests." (Aim, Fire -- Beck 2000)
TDD is, in part, an attempt at reducing the length of a feedback loop. The catch is that (in spite of the labels that have been used) the feedback loop of interest is not the programming-test loop, but instead the analysis-design-programming loop (bringing OOA, OOD, and OOP closer to each other).
The underlying assumption is something like "complicate code should be easy to test". If you believe that easy-to-test drives less-tightly-coupled design, and you think that latter characteristic is valuable, then it makes a certain amount of sense to lock in that easy-to-test constraint early.
"TDD strikes me as a practice that slows you down a fair amount yet still doesn't offer anything close to complete formal validation"
Yes and... that's not TDD's job? The automated checks used in TDD are written by the developer to satisfy the needs of the developer; if the thing you want is complete formal validation, then you should be using tools designed to meet that need. TDD might give you a higher success rate when you subject candidate systems to formal validation, and should give you lower costs when revising a failed candidate, but "the TDDs passed, ship it" is _not_ a risk free proposition.
Beck again: "I never said test-first was a testing technique. In fact, if I remember correctly, I explicitly stated that it _wasn't_."
(Also, TDD wasn't being hyped as a testing practice, but more of an analysis/design/programming practice: https://users.csc.calpoly.edu/~djanzen/setopics/testing/IEEE... )
Very roughly, Cucumber framework appears in late 2008? Whereas BDD first appears in the TDD community no later than December 2005.
Ex: here Dave Astels references a BDD talk he presented in 2005-09: https://web.archive.org/web/20051212160347/http://blog.davea...
Behavior Driven Development began as a re-languaging of TDD: "The developers were much more receptive to TDD when I stopped talking about testing." -- Dan North.
BDD diverged from TDD fairly early, after some insights by Chris Matts.
As for TDD advocating tests of the public interface... that seems to me to have been more aspirational than factual. The tests in TDD are written by developers for developers, and as such tend to be a bit more white/clear box than pure interface testing would suggest.
In the edge cases where everything you need for testing is exposed via the "public" interface, these are equivalent, of course, but there are tradeoffs to be considered when the information you want when running isolated experiments on an implementation isn't part of the contract that you want to be supporting indefinitely.
<< I call them "unit tests" but they don't match the accepted definition of unit tests very well. >>
I'm not entirely certain it's fair to accuse the author of lying; ignorance derived from limited exposure to materials outside the bubble (rather than deceit) is the more likely culprit here.
(Not helped at all by the fact that much of the TDD/XP origin story is pre-Google, and requires a different set of research patterns to track down.)
Retroactive continuity - how does it work?
For today's lucky 10,000: "unit test", as a label, was in wide use prior to the arrival of the Extreme Programming movement in the late 1990s. And the "unit" in question was the test subject.
But, as far as I can tell, _Smalltalk_ lacked a testing culture (Beck, 1994), so perhaps the testing community's definitions weren't well represented in Smalltalk spaces.
"The past was alterable. The past never had been altered."
(Not particularly fair to single out this one author - this origin myth has been common during the last 10 years or so.)
My guess: because it picked up a lot of momentum from the OO "patterns" community, and that gave it a lot of immediate reach and credibility.
Beck and Cunningham already had an audience of their own, and TDD also promoted Martin Fowler's ideas (pulling in his audience), and Robert Martin jumped aboard early as well (pulling in his audience).
The cynic in me notes that the Smalltalk community needed something to do, as it had by then become clear that Java was going to win that market.
Only viewed in isolation - the point of red-green is test calibration: be certain that your new experiment is actually measuring the thing that you think it is measuring.
Red-green isn't the only way to achieve this, of course, but it is a smooth way to achieve this.
Similar claims have been part of the TDD tradition since its very beginning.
So almost any site you hit with your browser is a "RESTful API". Another decent example is Atom Syndication / Atom Publishing.
The best introduction that I've found is Jim Webber's 2011 talk https://www.youtube.com/watch?v=aQVSzMV8DWc&t=3m8s Jon Moore's 2010 Hypermedia APIs talk is also good https://vimeo.com/20781278
Unfortunately, a second idea appeared, that embraced the label "REST" but used it to mean something else (roughly, a set of conventions for how to organize web pages and the edits one makes to web pages). And the second idea was written and shared with a much larger audience than the first (in addition to being an "easier" idea), so it grabbed a super majority of the mind share.
Use the same spelling conventions as the auto-generated messages seems a reasonable request.