Ironically, game controllers can have a latency of more than 20ms. Light travels more than 6000 km in a vacuum in 20ms. That's a pretty good excuse for gamers to complain about that sort of latency, I think.

Bluetooth is bad enough that videos have to be delayed so that audio can catch up. Completely worthless for games unless you use proprietary codecs like aptX.

I wouldn't say it's that bad, I flew a drone controlled via bluetooth with my phone, for instance.

Buffering audio is actually a great answer to disconnections. Plus, it's less energy-intensive to fill the buffers at once than to stream it as needed. And since it's pre-recorded anyway, you already have the whole buffer. Yes, initial transfer can take a while, though.

But in theory, the play/pause operations shouldn't have latency, and the bluetooth device could clearly communicate its state. I'm not sure how audio is actually implemented in the bluetooth stack, though, I've only ever done serial communication with it.

That's before taking into account enemy attempts to jam, spoof, or otherwise disrupt your control link.