> If you have 0.5s gap between audio and video, you hardly notice.
If your video comes later than audio and depicts human mouth talking you would notice it ever for 1 frame (0.02s for 50 frames/s) delay; but it's true that if audio is delayed relative to video, then 0.5s is hardly noticeable.
Think of it like playback vs a phone call. If you have 0.5s gap between audio and video, you hardly notice. But the same latency in a phone call makes it very awkward.
In music performance, the awkwardness would be multiplied by the number of spectators cringeing :)