My biggest concern is that test drivers can and have done the 'hard parts' of a lot of test routes, making the overall miles driven statistic kind of useless as a representation of the driving ability of the car.

But yeah, I'd agree there's a lot of difficulties expecting a test driver to immediately take over in a sudden event like a bike entering the roadway.