But I hear you complain "but that's so much work to maintain!!" - mocks are more work to test less, if you need a better fake often, write a library which has them. What you are referring to is known as fuzzing, not unit testing, and fuzzing has a better track record than straight up "testing" generally for finding issues. Fakes can have deterministic or indeterminate behavior, the case the OP makes is for deterministic "unit" testing with more real behaviors than not. This doesn't preclude good fuzzing...
"It's easier than one would think to accidentally rely on implementation details"
To be somewhat philosophical, what if that 'implementation detail' is the feature? No amount of testing will save you from incompatible feature sets, to 'enable testing of functionality that the current concrete implementation doesn't exercise' you run the very real risk of baking in incompatible sets of features into the core of your app. You shouldn't be designing your app at the unit level...
It is subtle and may not be worth the time, but again, it may be.
They explicitly call out clocks as a source of non-determinism that probably should be mocked, but I'll re-use them as an example anyway because everyone is familiar with them: it's extraordinarily useful for the tests to execute nearly immediately rather than actually waiting on a clock, and rare behavior like a clock running backward, two consecutive timings being identical within the clock's resolution, or whatever other weird artifacts that your code should handle are definitely better explicitly tested rather than not mocking the clock. Other domain-specific interfaces are often similarly able to exhibit a weird edge case that ought to be explicitly tested (rather than accidentally relying on a "nice" implementation) if you really want to unit test the callers and not integration test the coupled system.