Yeah, maybe there is an approach that works well with mocks but this has always concerned me as well. It's rare (for me anyway) that I want to test which calls a function makes rather than if the function returns the correct result.

I can imagine scenarios where that is useful and I know how to use mocks when that comes up ... but the examples are always a system where mock based testing is strangely coupled to implementation, very fragile and have the scary failure mode you describe.

When you're testing functions in an impure language (pretty much any language, short of extremes like Elm, maybe Haskell, etc), functions are used not only for their return value but also their side effects.

Side effects can be things like making an http request or writing to a file, it can be changing some global variable, but it can also be something like calling a public method of another public object. Since that function/method could also be impure, the very act of calling it is actually, in a way, part of the contract of the function under test: it returns a value, and it creates a side effect in the form of calling another function.

The only real way to test for those side effects is to pass in a stub or mock and to check that the function was called correctly. That makes your tests brittle, but it's the only way to make sure it does what you think it does (even if you only call the function for an intermediate return value, the fact that changing the intermediate function could change the result of the function under test).

Those tests are brittle because functions with side effects are, in fact, brittle, and favoring pure functions (where the only thing that matter is the return value, and dependencies are composed or passed as arguments) makes both your tests and your code better.

Typically, I find that I'm concerned with what calls a function makes when those calls interface things external to my program. I.e. I'm asserting that a request to a given http endpoint is made, or that an smtp client sends an email. I assume the external libraries work and am testing my interface to them.

This is where I find myself having to use these. Though, I greatly prefer to have the test merely check for the result of the call, if I can. e.g., if I have to post to a url to delete something, check to see if said something exists. In the end, it is not the call that I care about, but what the call was supposed to do.

This way can lead to brittle tests, clearly. But the reality seems to be that brittle systems lead to brittle tests. There is no testing panacea.

Why would it be of concern? I think mocks have a good place in unit tests. I generally believe the premise of unit tests is to test your code against specific pre-conditions. If those pre-conditions cannot be created in a unit testing environment then mocks are justified. I would, however, be very way of mocks moving past unit testing and into integration or system testing (or other varieties like failure injection testing, performance testing, etc.). Mocks have their uses & their limitations. Understanding that is key to using them well.

It means that you lose out on a lot of value from your test suite: besides verifying behavior on new code, tests also allow you to confidently refactor existing code. If your tests test implementation via mocking, and break when you refactor your code in ways that preserve the intended behaviors, then you're unable to confidently refactor.

Mocks lead you down a path where developers will eventually feel too scared or fatigued to refactor anything substantial.

> If those pre-conditions cannot be created in a unit testing environment

then you have technical debt to repay, by making those pre-conditions satisfiable