> Surely all the time is taken by the tests, not calling the test functions?

Calling tests has overhead. Also knowing how to schedule and parallelize tests with dependencies is not as "simple".