Oh I'd never considered lower power consumption as a reason for increased latency.

I don't immediately see the relationship however -- is there anything you could point me to, to explain the mechanism?

To have minimal latency basically everything needs to stay active constantly. CPU is constantly awake sending or receiving data. Radio is constantly awake. Interconnects constantly awake. If you instead send a large buffer, you can sleep between bursts of work. Dropping to lower power states in the interim, and only a small DSP doing the audio output needs to be awake.

Overhead that’s not linear with the buffer size. The cost (in CPU time, etc.) of updating a 480 sample buffer is similar to that of a 64 sample buffer.