News for hackers. Stuff that matters.
Re:Giving up on test-first development by ncphillips on Friday March 18, 2016 @07:10AM (#11311000)
> Because you generated the code being motivated by the tests, there are a lot of unit tests that dot test functional units - tests that are essentially testing implementation decisions.

I've seen this happen to myself numerous times as well, but I don't think this is good argument against TDD.

If a tests fails because the implementation details change, that tells us that we wrote a bad test, not that TDD is bad. As I get better at writing tests, I am better able to catch these kinds of errors ahead of time. There are a ton of little techniques you can use and questions you can ask yourself about your code that can make your tests better.

Furthermore, test code is just as important as production code, and we should be just as ruthless in refactoring and peer-reviewing each others tests (within reason).

TDD is NOT suitable for every problem, this is true. But I think too many people give up on it because they see "the problems of TDD" which are actually just "the problems of writing bad tests".

Note: I'm not trying to target you sago, but merely making a general observation. I'm talking more about people who don't write tests even close to the time of writing the code, or don't write any tests at all because the their tests kept breaking for no good reason and were slowing them down.

Re:Giving up on test-first development by sago on Friday March 18, 2016 @07:26AM (#11311043)
> that tells us that we wrote a bad test, not that TDD is bad

I strongly disagree. The only thing that matters to how software works for an end user is its boundary. You simply can't do TDD by only testing the boundary.

You have to write a test before each unit of code, so you have to test those internal interfaces, the way units of code work. But that is exactly the stuff that should be allowed to change. It's a mistake to assume 'implementation details' only exist within a function. That stuff you can avoid testing, but the behavior of the function is itself an implementation detail for whatever is using that function.

I simply cannot imagine TDD for anything other than a toy example where you don't end up writing tests that depend on the software's implementation, and are therefore inertia to changing that.

> we should be just as ruthless in refactoring and peer-reviewing each others tests

I strongly agree. But this is rarely part of the TDD ethos. Saying "I refactored the code and threw away 100 tests." is likely to send TDD folks twitching, in my experience.

Re:Giving up on test-first development by RHSeeger on Friday March 18, 2016 @08:57AM (#11311442)
> The only thing that matters to how software works for an end user is its boundary.

You're ignoring the fact that tests aren't only to validate "end user functionality". They're also there to make life easier for the developer; to make both initial development and maintenance easier.

If I need a custom sort algorithm for my product, I'm going to write unit tests for it's implementation. I'm also going write functional tests to make sure the end user is seeing things sorted. However, those tests serve different purposes.

The unit tests allow me to write clean, simple tests that show exactly what the implementation is supposed to be doing for different inputs. That allows me to change the implementation of the sorting algorithm later and worry less about breaking the functionality.

The functional tests are (generally) not going to be as clean and obvious, nor cover every possible partition of inputs that the implementation could get.

Re:Giving up on test-first development by sago on Friday March 18, 2016 @09:48AM (#11311727)
> You're ignoring the fact that tests aren't only to validate "end user functionality".

Erm, no, that was my entire point!

Re:Giving up on test-first development by RHSeeger on Friday March 18, 2016 @10:03AM (#11311822)
Wow... um... you're right. I was flitting around and dealing with external communication at the same time; you have my apologies.

I think my points align pretty well with what you were saying instead of against it.

Re:Giving up on test-first development by sago on Friday March 18, 2016 @10:27AM (#11311966)
Thanks for the response, it's fun to read someone strongly disagreeing while saying what you think. slightly baffling, but fun ;)
Re:Giving up on test-first development by agentultra on Friday March 18, 2016 @11:29AM (#11312398)
> If I need a custom sort algorithm for my product, I'm going to write unit tests for it's implementation.

I would start with writing an assertion about the end state of the sort. Then using syntactic substitution of the predicate calculi derive the program by assertions.

I'd then have a specification of the algorithm as a mathematical model from which I can check every possible execution for the entire domain on the sort. If I'm so inclined I would have a good basis for a proof.

The unit tests then become assertions on the implementation and don't have to verify correctness.

It would be awesome though if I could just generate the code directly from the specification!

Re:Giving up on test-first development by jkaptur on Friday March 18, 2016 @02:27PM (#11313777)
You would? Can you demonstrate how that would work for, say, a sort algorithm with a tunable "fuzziness" parameter? Informally, after the algorithm is complete, each element is within k indices of where it would be after a traditional sort. So k=0 could just be quicksort, but as k approaches the length of the input we have to do less and less work.

It's easy to dream up algorithms and traditional unit tests for this (and easy to see how the large input space makes the approach suboptimal). I'd love to see how your approach would work.

Re:Giving up on test-first development by MichaelBurge on Friday March 18, 2016 @08:38PM (#11316516)
Would you actually do this, or do you just like to imagine a world where people do this?

I'm familiar with Coq and Prolog(one of which I think you're hinting at), but I'd reach for Perl or C over either one if I needed to write a trading program or generate a statement from my accounting software or something.

Re:Giving up on test-first development by agentultra on Monday March 21, 2016 @12:30AM (#11326249)
I learning to use these techniques in my everyday work (cloud software).

I think of it like this: nobody is going to care if I screw up the script to balance my accounts. If I write some software that messes up the balance of everyone's account in some bank then someone is going to care.

I don't think formal methods and rigor are the sole domain of government institutions, avionics control software, and industrial robotics. The public relies on the cloud and the applications that run on it and they have no recourse if their data is exposed or disappears.

Imagine if instead of crossing over a bridge you were stopped at a booth and asked to sign a wager stating that nobody is liable if the bridge collapses underneath you while you cross it. Rich or poor, regardless of how much you pay in taxes, you should be able to rely on that bridge being built to the highest standards we know how and someone should be liable if it falls down. You can rely on this in real life. Why do we not have this for software?

I'm not saying I write all of my software using formal methods and rigorous proofs. It's often good enough to sketch my idea on the back of a napkin when I'm building a tool shed. But nobody built a sky-scraper without doing a little math and being fairly rigorous in their thinking.

Re:Giving up on test-first development by oldmanjay on Friday March 18, 2016 @10:05AM (#11311840)
I have used TDD to construct many production systems. Your lack of imagination is not a particularly compelling basis for an argument.
Re:Giving up on test-first development by sago on Friday March 18, 2016 @10:31AM (#11312002)
You're disagreeing with my 'I can't imagine'? So you're saying you've built non-toy systems with TDD where you've never tested anything but the user-facing behavior of the system?

I'd be interested in knowing how, and what kind of thing you build. I can almost imagine it for an API or framework, where almost every implementation choice is on the boundary. But even then, you have to test private functions to do TDD, surely.

I work on real time stuff, and in that stuff, the user-facing functionality is hard to test, and usually not tested in a TDD kind of way. The TDD stuff is all internal.

Re:Giving up on test-first development by mbrock on Friday March 18, 2016 @08:18AM (#11311240)
Changing tests is work, yes. Unit tests are precisely about verifying implementations, and maintaining them is the price you pay for having these layers of verification. It's inertia in the way a safety net is a barrier.
Re:Giving up on test-first development by hinkley on Friday March 18, 2016 @06:09PM (#11315597)
I have always found the 'safety net' analogy uninspiring, and people who think they're tough and manly don't need safety nets, thankyouverymuch.

Now, safety equipment in a race car is something else entirely. Half of it keeps you from dying immediately on impact, sure, but the other half actually lets you go faster. The 5 point harness, for example, helps keep you from losing control of the vehicle during high G maneuvers, by keeping your torso in the seat and therefore your legs and arms in (roughly) the correct orientation to the steering wheel and the pedals.

Re:Giving up on test-first development by firepoet on Friday March 18, 2016 @08:35AM (#11311313)
Not this TDD folk. :-)