News for hackers. Stuff that matters.
Re:Replacing WebRTC: real-time latency with WebTransport and... by tamimio on Monday October 30, 2023 @11:17AM (#38070504)
I got excited for a second that a new something will replace webrtc for media/videos.. it is not. Around 2 years ago in a project I wanted to transfer a raw 4K stream with low latency (sub 50ms) over cellular network, WebRTC performed poorly it was a no go, I ended up making my own algorithm that enables FEC (Forward Error Correction) based on IETF payload scheme 20, and piped it through UDP with Gstreamer and managed to make it work, but obviously wasn’t in the browser.
Re:Replacing WebRTC: real-time latency with WebTransport and... by englishm on Monday October 30, 2023 @12:13PM (#38071484)
That sounds like it was an interesting project! Jean-Baptiste Kempf also presented something at Demuxed last week using FEC and QUIC datagrams to get similarly low latency delivery over WebTransport into a browser. There's also a draft[1] for adding FEC to QUIC itself so it's quite possible Media over QUIC (MoQ) could benefit from this approach as well.

I'm not sure why you say "it is not." We have a working MoQ demo running already on https://quic.video that includes a Rust-based relay server and a TypeScript player. The BBB demo video is being ingested using a Rust-based CLI tool I wrote called moq-pub which takes fragmented MP4 input from ffmpeg and sends it to a MoQ relay.

You can also "go live" from your own browser if you'd like to test the latency and quality that way, too.

[1]: https://www.ietf.org/archive/id/draft-michel-quic-fec-01.htm...

Re:Replacing WebRTC: real-time latency with WebTransport and... by tamimio on Monday October 30, 2023 @12:55PM (#38072096)
Thanks, it was a big project and streaming 4k in realtime from a flying drone was one of the challenging parts! I have some write up about it although nothing too technical but some videos there demonstrating some differences(1)

> I'm not sure why you say "it is not."

Pardon my ignorance it looked like it isn’t replacing WebRTC entirely yet, but glad I was wrong, I never tried anything QUIC for media related, would love to try the MoQ tool you did, and like the fact it’s rust based too as the one I did was written in rust. I will give it a test for sure, it’s been two years and I wasn’t following any updates so hopefully there’s an improvement compared what it was back then.

> takes fragmented MP4 input from ffmpeg and sends it to a MoQ relay.

Just a quick question, is ffmpeg a “requirement” per se for that CLI tool? As I remember I had to ditch ffmpeg in favor of gstreamer since the former one was eating up a lot of resources compared to Gstreamer, and it was crucial issue since the server was basically an SBC on a flying drone.

(1) https://tamim.io/professional_projects/nerds-heavy-lift-dron...

Re:Replacing WebRTC: real-time latency with WebTransport and... by englishm on Monday October 30, 2023 @01:19PM (#38072486)
The current moq-pub implementation only requires valid fMP4 (a la CMAF) to be provided over stdin. I haven't tested, but I imagine you can do the same with gstreamer.

Separately, I've been working on a wrapper library for moq-rs that I've been calling 'libmoq'. The intent there is to provide a C FFI that non-Rust code can link against. The first integration target for libmoq is ffmpeg. (I have a few bugs to work out before I clean up and shout about that code, but it does mostly work already.)

I gave a presentation about some of this work last week at Demuxed, but the VoDs probably won't be available on YouTube until Decemberish.

Also, I understand the gstreamer project has better support for Rust so I'll be looking at that soon, too.

Re:Replacing WebRTC: real-time latency with WebTransport and... by oplav on Monday October 30, 2023 @07:44PM (#38077601)
I believe gstreamer has support for fMP4: https://gstreamer.freedesktop.org/documentation/fmp4/index.h...

I stumbled upon a MoQ video by kixelated a couple months ago and have been meaning to give the above a try, but haven't gotten around to it yet so not sure if it will do the trick.

Re:Replacing WebRTC: real-time latency with WebTransport and... by tamimio on Thursday November 02, 2023 @12:50AM (#38109132)
Appreciated! Where do I find these presentations, if you don’t me asking?
Re:Replacing WebRTC: real-time latency with WebTransport and... by englishm on Friday November 10, 2023 @08:02AM (#38218299)
https://2023.demuxed.com/ has the day-long uncut videos available behind a $20 "VoD-only" post-event "ticket" paywall.

Eventually https://www.youtube.com/@Demuxed will have the edited VoDs for Demuxed 2023 available for free. Usually those go up around December-ish.

I also posted my slides on LinkedIn: https://www.linkedin.com/feed/update/urn:li:activity:7124766...

Re:Replacing WebRTC: real-time latency with WebTransport and... by Sean-Der on Monday October 30, 2023 @12:15PM (#38071522)
That's a cool problem!

I could see how WebRTC out of the box would perform poorly. It wants to NACK + delay to give a good 'conferencing experience'.

I bet with FlexFEC [0] and playout-delay [1] you would get the behavior you were looking for. The sender would have to be custom, but the receivers (browsers) should just work! If you are interested in giving it a shot again would love to help :)

[0] https://datatracker.ietf.org/doc/html/rfc8627

[1] https://webrtc.googlesource.com/src/+/main/docs/native-code/...

Re:Replacing WebRTC: real-time latency with WebTransport and... by tamimio on Monday October 30, 2023 @01:04PM (#38072235)
It was an interesting problem indeed! I had some write up about it (and the whole project) in a link I have in the comment above, might gives more context.

> I bet with FlexFEC [0]

The previous draft (1) of this was my basis when I did the FEC sender. I didn’t manage to have it streamed 4K into the browser though, my client was OBS with gstreamer as it was far more performant than a browser in my tests, have you/any demo you did that I can try to stream it into the browser? That would be really major improvement! And appreciate the help, O would definitely give it another shot!

(1) https://datatracker.ietf.org/doc/html/draft-ietf-payload-fle...

Re:Replacing WebRTC: real-time latency with WebTransport and... by Sean-Der on Monday October 30, 2023 @01:27PM (#38072581)
I would love too! Join https://pion.ly/slack and I am Sean-Der

If you prefer discord there also is a dedicated space for ‘real-time broadcasting’ https://discord.gg/DV4ufzvJ4T

Re:Replacing WebRTC: real-time latency with WebTransport and... by tamimio on Thursday November 02, 2023 @12:48AM (#38109116)
Thank you!
Re:Replacing WebRTC: real-time latency with WebTransport and... by imtringued on Monday October 30, 2023 @01:16PM (#38072440)
Well, a raw 4k stream has a bit rate of 11.9 gbps. I would be surprised if that worked over a cellular network at all.
Re:Replacing WebRTC: real-time latency with WebTransport and... by tamimio on Monday October 30, 2023 @01:26PM (#38072557)
I think it was around 3gbps or even less if I remember correctly, it wasn’t 60fps and I can’t remember the color depth either, and the cellular network was a SA (Stand Alone) mmwave private network, not a commercial one, so it did work in our project eventually.
Re:Replacing WebRTC: real-time latency with WebTransport and... by vlovich123 on Monday October 30, 2023 @01:24PM (#38072531)
At 60fps but yeah, I think they mean they still passed the raw 4k stream through a lossy codec before putting it over cellular.
Re:Replacing WebRTC: real-time latency with WebTransport and... by modeless on Monday October 30, 2023 @03:30PM (#38074495)
Yeah. From what I can see WebRTC is basically a ready made implementation of a video calling app, bolted on the side of the browser, that you can slap some UI on top of. If you want to do anything that isn't a straight Zoom style video calling app (or Stadia), you'll immediately run into a wall of 5 year old known but unfixed bugs, or worse.

To be fair, video calling is an incredibly important use case and I'm glad it can be done in the browser. I just hope that WebRTC really can someday be replaced by composing multiple APIs that are much smaller in scope and more flexible to allow for use cases other than essentially Zoom and Stadia.

Re:Replacing WebRTC: real-time latency with WebTransport and... by est on Monday October 30, 2023 @10:11PM (#38079305)
> WebRTC is basically a ready made implementation of a video calling app, bolted on the side of the browser

Yeah, Google bought GlobalIPSound, dumps the source code to w3c as a standard.