Re:Replacing WebRTC: real-time latency with WebTransport and...
by padenot on Monday October 30, 2023 @01:57PM
(#38073112)
https://blog.paul.cx/post/audio-video-synchronization-with-t... has some background, https://github.com/w3c/webcodecs/blob/main/samples/lib/web_a... is part of a full example that you can run using web codecs, web audio, audioworklet SharedArrayBuffer, and does A/V sync.
If it doesn't answer your question let me know because I wrote both (and part of the web audio spec, and part of the webcodecs spec).
Re:Replacing WebRTC: real-time latency with WebTransport and...
by kixelated on Monday October 30, 2023 @02:24PM
(#38073538)
I'm using AudioWorklet and SharedArrayBuffer. Here's my code: https://github.com/kixelated/moq-js/tree/main/lib/playback/w...
It's just a lot of work to get everything right. It's kind of working, but I removed synchronization because the signaling between the WebWorker and AudioWorklet got too convoluted. It all makes sense; I just wish there was an easier way to emit audio.
While you're here, how difficult would it be to implement echo cancellation? The current demo is uni-directional but we'll need to make it bi-directional for conferencing.
I'm still just trying to get A/V sync working properly because WebAudio makes things annoying. WebCodecs itself is great; I love the simplicity.