News for hackers. Stuff that matters.
Re:Still no consensus on testing private methods by hinkley on Thursday March 10, 2022 @04:04PM (#30632411)
There's an inflection point with unit tests where a good description, a good matcher, and a small number of preconditions (mostly handled by nested test suites) make it easier to rewrite the test than to fix it. When I say 'cheap' tests, these are what I mean. Tests you can treat like livestock instead of pets.

The sentiment of the test may still apply, but the implementation details mean you're testing, for instance, for empty set using a different strategy. Block delete and start over.

Or the feature you're adding is meant to allow us to deal with empty set: let our customers put things in their shopping cart even though we don't have any shipping addresses for them. In that case we 'broke' that requirement, and just delete the test and move on.

But getting cheap tests and high coverage is a bit of an art, often learned via the school of hard knocks. If you let too many requirements leak up into integration tests, you may find you have many, many tests to fix when a "never" or "always" changes to "occasionally", "mostly". Some people get brave and delete these tests, and discover Chesterton's Fence later when someone yells at them for deleting regression tests, because the test described 2 scenarios it was asserting, but it surreptitiously asserted 2 more, one of which the New Guy just broke but got a green build.

With a unit test it's tenable to make a test that looks like it checks one rule while actually checking only one rule.

Re:Still no consensus on testing private methods by travisjungroth on Thursday March 10, 2022 @05:36PM (#30633529)
See, I think your example of breaking the thing is where things are tricky. I know you’re not advocating for it, but I feel like a lot of people get burned by an experience like that. They think “Someone wrote a test. Then it broke. Then someone deleted it or fixed it wrong. Then something else broke and no one noticed because the test was deleted. Therefore, don’t write too many tests.”

Isn’t that last step super weird? I don’t see how not writing tests would have prevented the regression. At most vague hand wavey “people would be more careful” which I don’t buy. The problem was breaking a dependency. Tests are not themselves dependencies, they are an automated stand-in for them.