It's certainly interesting because they can have a huge effect on the measurement results. For example, with HDA and I suspect other audio codec systems, what you described can be done with essentially 0 delay by configuring the codec to do the in/out mix itself. If you are measuring output from application buffers to input to application buffers, then how big they are and at what sample rates and bit depths will also affect the latency you see, etc.

Certainly nobody would then agree this is measuring the latency of the audio _system_, it would only measure the latency of the audio device, right?

I think we can give the benefit of the doubt and assume they are not trying to actively deceive.