This is spot on IMO. I often see people touting the benefits of unit tests during a refactoring...but 95% of the time refactoring involves modifying class APIs since the hardest part of development is getting the object model right. Unit tests only assist refactoring when you don't modify the APIs - in other cases they are a burden.

Yes, if you change a unit's interface you will have to also change any code relying on said interface. That's a maintenance cost of unit tests.

But it doesn't follow that changing a unit's interface means unit tests suddenly become just a burden. Ideally unit tests are, well, testing a bunch of core functionality of the unit under test. You adapt them to the new interface. Then you're back to having a quick, automatic sanity check you can run against the unit whenever you have to make a change.

I don't understand people bemoaning this 'cost' of unit tests when the benefits they provide typically far outweigh the costs. It's possible broader functional/integration tests have a better ROI in certain situations, but they come with a maintenance cost as well.

>But it doesn't follow that changing a unit's interface means unit tests suddenly become just a burden.

If your refactoring is largely centered around changing unit interfaces (not uncommon) then it means that those unit tests are 100% overhead because most of the time they fail just because you changed the code.

>I don't understand people bemoaning this 'cost' of unit tests when the benefits they provide typically far outweigh the costs. It's possible broader functional/integration tests have a better ROI in certain situations, but they come with a maintenance cost as well.

I've certainly found that the ROI is a lot better. I find that integration tests have a higher up-front cost but maintenance-wise they're the same or cheaper. % of failures that are actually catching bugs is higher too.

I mostly agree, except that for parts of the code that meet these conditions, I aggressively unit test as the default. To the point that if an end-to-end test uncovers a potential bug in this component, my first action is to write a new unit test for this component inspired by the end-to-end test. If that unit test fails, then I can focus on just the unit test, and not worry about the end-to-end test until the unit test passes.

Another condition I would add to the list, which the component of mine I am thinking about meets is:

5) Failures in this component have cascading effects to multiple other components, causing seemingly "impossible" failures that are not obvious to others that they were caused by this component.

If your end to end test uncovers a bug in a lower level module that has a clean and stable abstraction, yes it makes sense to write a test that surrounds that instead.

That needn't be a unit test. It could simply be a lower level integration test.

In some cases maybe, but it's not effective in my case. In order to catch failures in this particular component, the most effective thing is to check that a large set of invariants is still true. An integration test of some kind only checks for behavior; does this component work when interacting with the rest of the system? That can show a failure exists, but is not useful for showing why. It's the unit tests which continually check the large set of invariants that can really get to the why: if a particular invariant is no longer true, the path to the problem is usually clear.

Maybe that should be a number 6): easily tested invariants.

Reading this makes me realize I had a bias against making lower level integration tests in this situation, definitely something I'll have to look for in the future.