vr000m

I think being able to code by talking is going to be the future. However we need to figure out how to switch between different projects and have it do what I mean and not what i say.

Speech is a lot more imprecise and perhaps less coherent but we can speak more than we can type. And I think an extra LLM step to cleanup what was said and anfew turns for LLM to clarify what was said. Can really unlock the potential.

The latencies in the table are based on heuristics or averages that we’ve observed. However, in reality, based on the conversation, some of the larger latency components can be much lower.

Daily now supports dial-in and dial-out, https://docs.daily.co/guides/products/dial-in-dial-out#main

Which means you can connect a bot to a call, and tell it dialout to a phone number and it will.

High level overview of how video is delivered to 100,000 participants using mesh/cascaded video routers (aka, SFUs) with 200ms latency.

Servers are geographically spread, servers coordinate with each other to route media. Selectively forward media based on who is watching whom in the viewport and at what quality.

Reminds me if the work from @lkozma in 2007, http://www.lkozma.net/wpv/

Agreed, all of the early HTTP based streaming was about HTTP progressive download. Microsoft’s smooth streaming was dominant in the early 2000s used it too. Glad that Twitch has been using it.

For me, DASH and HLS are just manifests or playlist definitions, of how best to find the resource. It is not dissimilar to webrtc signalling, i.e., go to a resource to discover where to get other parts of the resource.

Agreed and thank you for the contribution. Alas, the work is now embroiled in breaking it apart into smaller parts.

Unfortunately modern routers made multicast unrounable, otherwise, we’d all be using mbone

All the listed protocols came after HTTP. RTSP, SIP borrowed heavily (albeit badly in retrospect) from HTTP.

I do not have all the historical context (early 90s), but for WebRTC, the idea was to not define any new protocol(s) or do a clean slate design. but rather to just agree on the flavors of the various protocols, and then to universally implement those. We already had SDP, RTSP, RTP, SAP, etc. And the idea was to cobble together the existing protocols into something everyone could agree on (the young companies, the old companies, etc)

We ended up defining variations to the flavors that we already had and for the most part everything turned out okay (maybe the SDP plan wars did not end up where we wanted it, but… it was a good enough compromise).

For realtime media, if we are able combine “locator:identifier” issue, we will be able to make media and signaling work inband.

Pavlov’s comment is correct. I came to add that soon the stream can be stored on customer’s own S3. Ergo, you’d be able to do a call in real-time, store it on your S3 account and make it available for streaming.

On your own S3, this would be a multipart upload.

In the case of QUIC, it is likely that the streaming would be over H/3 (HTTP3) or HTTP over QUIC. They may fallback to H1 or H2 but typically over a long enough time, firewall rules become more relaxed.

Most common browser implementations use an Open GOP. That means an IFrame is implemented when needed. On scene change or when there’s high motion.

Only naive implementations would burst an IFrame on to the network, most pace them. And if needed, you could split your iframe into several frame intervals and decode them without creating a burst by bit rate.

Actually a lot of webrtc implementations use 1s or 2s GOP length. Again depends on how much control you’ve on your pipeline. Browsers implementations do make some assumptions on usecase.

Webrtc protocol doesn’t dictate 1 or 2. Although browsers do implement some of their own assumptions for this. By default the client side buffer can be orders of 100s of milliseconds. this is as you pointed out tuned for real-time or live applications.

If you’re doing something like YouTube/Netflix and want to avoid going to a lower definition of the stream, that too can be tuned, albeit you’d want to use simulcast and implement your own player (to feed the video and audio frames for decoding at the pace you dictate).

ICE is needed when both parties are NATes, if one party was not mated, we’d not need ICE in webrtc either.

Agree on 2.

On 3. The videobridge needs state on who is on the session and who to forward to. that requirement doesn’t go away with QUIC. Unless you’re thinking that the video streams are some kind of a named resource or object.

I think the most people gripe about is SDP and it’s prescription of negotiation and encoding. I agree that capability negotiation can be vastly simplified given some of the capabilities can be inferred later in the session.

A future blogpost will talk about all the networking work that went into optimizing for these large size participation.

Webrtc uses a few protocols. RTP is very central to it, ICE and SDP are also very important protocol for NATs/firewalls and capability selections. While we use webrtc as a shortcut to refer to the suite of protocols and didn’t call out RTP specifically for that reason.

Agreed. My calculations are totally skipping finding/updating the manifests or exchanging the SDP Offer/Answers.

Media over QUIC is interesting. For RTP or peer to peer QUIC, there is more work to be done. But you will end up engineering many of the same things as webrtc suit of protocols (ICE -- STUN, TURN, MULTIPLEXING, etc).

QUIC and webtransport can definitely already do DASH/HLS without some of the protocol complexity by using the QuicStreams (but to use QUICs underlying features, DASH/HLS need to change as well).

Some of us wrote a position statement in 2017, see https://datatracker.ietf.org/doc/html/draft-rtpfolks-quic-rt.... There are new documents around media ingest being proposed currently.

Low latency HLS is creating partial segments by bucketing 200ms of frames instead of 6s segments in standard HLS. Whereas in webrtc, the endpoint is sending the frame as soon as it is ready.

The apples apples comparison here is 0ms (in webrtc, no send side buffering) vs 200ms (in low latency HLS) or 6s (in standard HLS). This is independent of latency of the endpoint from CDN or source.

Another distinction is playback wait time, i.e., how quickly upon joining can it start rendering video.

I’m assuming the full reference picture (typically, an IFrame or a golden frame depending on the codec) in low latency HLS is only available at the start of each 6s segment and not in partial segments. So joining a live stream, the receiving endpoint would have to wait at most 6s before rendering.

Similarly in webrtc, it’s up to the system to generate a reference frame at regular intervals, as low as every second. Or to do it reactively, a receiving endpoint can ask the sender to send a new reference picture. This is done via a Full intra request, the wait time can be as quick as 1.5 times of the round trip time (as new codecs can generate a new iframe instantaneously upon receiving a request). There’s a slight cpu penalty for this which means that the sender getting too many full intra requests may typically throttle the response to 1s.

So Apples to Apples comparison for wait time would be up to 1s for webrtc vs 6s for HLS.

One thing to consider is if you would have One API and a few knobs to control the experience and not have to worry about the underlying protocol-, reliability-, latency- aspects to build that experience. (particularly not worry about the shenanigans around tuning RTMP, RTP, HLS, or webrtc)

There are some use-cases in every industry. * Finance/Company: earnings calls as mentioned before, all-hands meetings at companies (with several people queuing for questions and answers) * Live Events (music or talkshows) * Tutoring or education in general. * Healthcare and education -- Surgeries which are broadcast to several schools and have active collaboration from some doctors. The large-scale interaction can begin much earlier before the surgery.

This was set up in 2019, the team having some fun! Thanks for finding us.

We do have a free plan, it’s not publicly listed at the moment. If you do under 20 000 minutes or so a month, it should not prompt for a credit card.