You could divide (in your head) contract and implementation details. Contract - what "unit" should do, e.g. factorial function should compute factorial for given numbers. But how exactly this is done is implementation detail (will be used recursion? do/while loop? will there be result caching?)

The trick is to omit implementation details from testing, because implementation can change, but after all fact(4) should return 24 no matter what.

So it's useful to think in categories of contract given unit should fulfill instead of testing everything for the sake of testing.