Superior Testing: Stop Stopping

Re:Superior Testing: Stop Stopping by skohan on Thursday January 31, 2019 @04:35AM (#19043603)

> It is the goal of every competent software developer to create designs that tolerate change [...] Code without tests is bad code. It doesn’t matter how well written it is; it doesn’t matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don’t know if our code is getting better or worse.

I agree that testing is a valuable tool for making good software, but I think the idea that all code categorically requires testing to be considered good is overzealous.

I think the most ardent TDD proponents underestimate the costs of testing. Tests are also code which can have bugs and has to be maintained, and having a well-developed test-suite can act as a type of inertia which makes it harder for an organization to make necessary architectural changes.

Don't get me wrong - I think testing is important, but it is just one tool which has to be balanced against a number of other factors.

[ Parent | on HN ]

Re:Superior Testing: Stop Stopping by pjbster on Thursday January 31, 2019 @04:46AM (#19043641)

I have two anecdotes about testing. First one was where I decided to refactor a function which had something like a 10-way cascading branch each of which had a compound test. Normally I wouldn't bother but, on this occasion, I wrote something like 60 unit tests in preparation for the exercise and one of those tests exposed a misplaced closing parenthesis in my solution.

The second case was where an in-house quotes engine was to be migrated to a SOAP service and some calculations needed to be ported. We didn't have access to the source so we created a small set of the most complex scenarios we could come up with and used those to generate calculator requests. I think we had 26 or 27 test cases and they each required non-trivial setup before the calculator could be invoked. Those cases exercised code which took the developer about 3 months to refine into a working solution.

So what does this reveal? I don't know. On the one hand, we had just under 60 unit tests which picked up 1 bug whilst, on the other, we had less than half that number of end to end tests which were sufficient to build a major piece of business functionality.

My gut feeling is that end to end testing is a better long term investment and unit testing is perfect for refactoring but inefficient for anything else.

[ Parent | on HN ]

Re:Superior Testing: Stop Stopping by pacala on Thursday January 31, 2019 @11:45AM (#19046112)

Many of us had to deal at some point in our careers with 'legacy ball of mud with no tests' codebases. Furthermore, longer lived products require a certain degree of inertia; it is hard for users to adapt to what appear to be arbitrary changes on a regular basis. Reasonable to understand why so many people are touchy around the subject of testing.

That being said, there is a certain degree of intellectual rigidity when it comes to TDD. The point of a good test suite is to capture the behavior of the product from a user perspective, and not to duplicate implementation details, or test the same behavior at three different levels of the architecture.

Ideally, each path in the code should be exercised by one [and no more] test written from the perspective of the product, but not necessarily using the user-visible APIs. To exemplify test refrain, suppose one is building an IDE. Write one test that the IDE frontend surfaces errors from the compiler to the user, and write a test suite for the compiler to check they produce sensible errors, using code coverage tools to make sure all possible errors are accounted for. But don't test each and every compiler error using the IDE frontend APIs.

[ Parent | on HN ]