Re:MoQ: Refactoring the Internet's real-time media stack
by englishm on Friday August 22, 2025 @04:57PM
(#44989720)
Synchronized playback is usually primarily a player responsibility, not something you should (solely) rely on your transport to provide. We have had some talk about extensions to allow for synchronizing multiple tracks by group boundaries at each hop through a relay system, but it's not clear if that's really needed yet.
Essentially though, there are typically some small jitter buffers at the receiver and the player knows how draw from those buffers, syncing audio and video. Someone who works more on the player side could probably go into a lot more interesting detail about approaches to doing that, especially at low latencies. I know it can also get complicated with hardware details of how long it takes an audio sample vs. a video frame to actually be reproduced once the application sinks it into the playback queue.
Re:MoQ: Refactoring the Internet's real-time media stack
by kixelated on Friday August 22, 2025 @04:51PM
(#44989653)
Each track is further segmented into streams. So you can prioritize new > old, in addition to audio > video.
Re:MoQ: Refactoring the Internet's real-time media stack
by madsushi on Friday August 22, 2025 @04:53PM
(#44989669)
Depending on the streaming protocol (eg WARP), you can specify that the tracks (audio vs video) need to be time-aligned, so each group (chunk of video or audio) starts at the same time and lasts the same length. I think this means you'll get resync'd at the start of the next group.
Re:MoQ: Refactoring the Internet's real-time media stack
by vlovich123 on Friday August 22, 2025 @10:32PM
(#44992530)
If you’re delivering audio and video separately the blocking is irrelevant for needing to solve synchronization. That’s why some amount of buffering (a few frames of video at least) on the receiver is needed to hide the jitter between packets / make sure you have the video. You can go super low latency with no buffering but then you need to drop out video / audio when issues occur and those will be visible as glitches - depends on how good your network is.
It is likely that I am missing this due to not being super familiar with these technologies, but how does this prevent desync between audio and video if there are lost packets on, for the example, the audio track, but the video track isn't blocked and keeps on playing?