Thus it is often a test of the implementation's assumptions when first written, and even worse, when the code is maintained/edited, the test is merely changed to get it to pass, because unit tests with mocks are usually:
1) fragile to implementation 2) opaque as to intent
Whereas input/output integration points are more reliable, transparent, and less fragile to implementation changes if the interface is maintained.
However, if you must do mock-level interaction testing, Spock has made it almost palatable in Javaland.
This is one area where functional fans get to make the imperative folks eat their lunch.
printf("hello world");
And the test is: mock_printf(string) {
if string != "hello world" then fail;
}
Which is basically just duplicating your code as tests.Usually because interfaces need to be clearly/well defined and overridden... In JS there are tools (rewire, proxyquire, etc) that can be used with tests in order to easily inject/replace dependencies without having to write code to support DI nearly as much. In fact, I'd say that there's usually no reason not to be very close to 100% coverage in JS projects.
On .NET it is a plain library, which requires the VS Ultimate editions to be useful and on Java it was mostly third party libraries.
C# design team is considering adding proper support as language feature, but it seems to be a very far away feature still.
C++20 might get contracts, but C++17 just got ratified, so who knows.
D, Ada 2012 and SPARK do already support contracts.
http://dlang.org/spec/unittest.html
It's been very effective at improving the overall quality of the code. And because of CTFE (Compile Time Function Execution) and static assert, many code correctness tests can even be run at compile time.
I wonder what that would look like...
We should go a lot further though. IMO, a unit that does not pass a spec/test should cause a compile time error. Testing systems should facilitate and converge with formal verification. Where possible, property based testing should be used and encouraged. And debugger tools should be able to hone in on areas where the result diverges from expectations.
We can achieve this in dependently-typed languages like Idris. First we define a datatype 'Equal x y', which will represent the proposition that expression 'x' is equal to expression 'y':
data Equal x y where
Refl : (x : _) -> Equal x x
There are two things to note:- There is only one way to construct a value of type `Equal x y`, which is to use the constructor we've called `Refl`.
- `Refl` only takes one argument, `x` (of arbitrary type `_`), and it only constructs values of type `Equal x x`. This is called "reflexivity", thus the name `Refl`.
Hence if we use the type `Expr x y` anywhere in our program, there is only one possible value we can provide that might typecheck (`Refl x`), and that will only typecheck if `Expr x x` unifies with `Expr x y`, which will only happen if `x` and `y` are the same thing; i.e. if they are equal. Thus the name `Equal`.
This `Equal` type comes in the standard library of all dependently typed languages, and is widely used. To use it for testing we just need to write a test, e.g. `myTest`, which returns some value indicating pass/fail, e.g. a `Boolean`. Then we can add the following to our program:
myTestPasses : Equal myTest True
myTestPasses = Refl myTest
This will only type-check if `Equal myTest myTest` (the type of `Refl myTest`) unifies with `Equal myTest True`, which will only be the case if `myTest` evaluates to `True`.[0] https://developer.salesforce.com/docs/atlas.en-us.apexcode.m...
And another example of something I'd want checked by the language is exception throwing / handling. It's another one of those places where code coverage won't help you unless you already know what you're looking for. Languages are getting better about it, but in general, handling errors is hard.
In haskell you embed domain specific languages when you require side effects:
class MonadState s m where
get :: m s
put :: s -> m ()
This is like an interface specifying the basics of all stateful computations.You can use different implementations for production and testing without changing any code so mocking is built into everything.
One other thing about code coverage, unit testing, and other testing fads is I think they actively affect the architecture, and usually in the overthinking it way.
Instead of having one tight bit of procedural code (which may have some state, or some dependency calls), people split it up into multiple classes, and then test each bit. This allows them to use the class architecture to mock things, but really has just multiplied out the amount of code. And in the end, you're running tests over the golden path probably even less. It's even possible to have 100% of the code covered, and not run the golden path, because you're always mocking out at least one bit.