News for hackers. Stuff that matters.
Re:OBS Merges WebRTC Support by Sean-Der on Saturday June 10, 2023 @02:07PM (#36273080)
WebRTC support has been merged into OBS. This is going to bring some really exciting new things to the space!

* Serverless Streaming - WebRTC is P2P so you can video right into your browser. You don’t have to stand up a server anymore to stream for a small audience.

* Sub-Second Latency - Create content and interact with viewers instantly. There is something magical about having a real conversation with your viewers.

* Multitrack Input - Upload your transcodes instead of generating then server side. Give viewers multiple video tracks to see action from all sides.

* Mobility - WebRTC lets you switch networks at any time. Go from WiFi -> Mobile with zero interruptions.

Try it out today with Broadcast Box. A reference server implementation. https://github.com/glimesh/broadcast-box. The PR to add WebRTC to OBS https://github.com/obsproject/obs-studio/pull/7926

-----

So many people went into making this happen. This was a colossal undertaking that different people have working on for over 6 months.

Sergio Murillo - Created WHIP. The only reason this could be added to OBS.

Luke Strickland - Created Broadcast Box. The reference server we developed against

Paul-Louis Ageneau - Creator of libdatachannel. The library used to add WebRTC support to OBS.

Colin Edwards - Started the project to add WebRTC into OBS

John Bradley, tt2468, pkv - Developers who worked on the code itself

RytoEx, tytan652 - Lots of feedback and reviews

-----

Have fun using it! If you have any questions/feedback/improvement ideas I would love to hear.

Re:OBS Merges WebRTC Support by EGreg on Saturday June 10, 2023 @02:39PM (#36273521)
Traveling in Europe currently, but got curious.

How many viewers can it support? RTMP usually goes to platforms that broadcast to many users. What about with WebRTC, can I have 100 viewers peer to peer? Maybe with TURN relays?

Re:OBS Merges WebRTC Support by charcircuit on Saturday June 10, 2023 @02:50PM (#36273645)
That's an apples to oranges comparison. With proper infrastructure both can scale to many users. For streaming to many people peer to peer will be inefficient, so I would recommend not doing that.
Re:OBS Merges WebRTC Support by EGreg on Saturday June 10, 2023 @02:55PM (#36273717)
How can OBS with the new WebRTC and WHIP+WHEP be used to stream to thousands of people at once, without using RTMP?

And what is the proper infrastructure to scale WebRTC to thousands of listeners at once?

Re:OBS Merges WebRTC Support by fidotron on Saturday June 10, 2023 @03:19PM (#36274036)
In practice this is what SFUs are for. You would simulcast a set of different encodings from OBS to the SFU and that would fan out to other SFUs and clients as appropriate until the desired scale is reached. This does get expensive.

If you don’t actually need the low latency then you are better off with hls-ll delivered via a CDN.

Re:OBS Merges WebRTC Support by EGreg on Saturday June 10, 2023 @03:49PM (#36274355)
If I'm ingesting and decrypting the stream anyway, wouldn't it be better to build an MCU, to send out one stream and adapt the resolution for each client?

https://www.digitalsamba.com/blog/p2p-sfu-and-mcu-webrtc-arc...

Re:OBS Merges WebRTC Support by fidotron on Saturday June 10, 2023 @04:04PM (#36274505)
Cost. SFUs don’t have to transcode the video, just pass buffers around and ensure both ends agree on the format. That is an enormously lighter process.

Personally I think simulcast (sending multiple differently encoded streams) is almost always wrong for many Webrtc uses (especially mobile publishers) but in the OBS case it actually makes a lot more sense.

Re:OBS Merges WebRTC Support by EGreg on Saturday June 10, 2023 @04:15PM (#36274615)
How does WebRTC know what resolution to send? Or does it always send the original resolution?
Re:OBS Merges WebRTC Support by fidotron on Saturday June 10, 2023 @04:20PM (#36274674)
That all depends on how you set it up. Generally speaking a lot of WebRTC is about how the connection negotiates the parameters of the delivery of the different media, including codecs, resolution, etc. One track can be delivered over a connection in multiple different ways at once as well.

If you want to know more you're probably best off going through the MDN docs about the WebRTC browser API and learning from that.

Re:OBS Merges WebRTC Support by Sean-Der on Saturday June 10, 2023 @03:17PM (#36274005)
I have talked with companies that are already doing 100k+. I don't have the exact numbers but Millicast and Phenix both do very large audiences.
Re:OBS Merges WebRTC Support by EGreg on Saturday June 10, 2023 @03:32PM (#36274178)
How are they doing it is the question? What's the set up?
Re:OBS Merges WebRTC Support by grogenaut on Saturday June 10, 2023 @03:15PM (#36273990)
How can a webserver serve 10k people at once?
Re:OBS Merges WebRTC Support by charcircuit on Saturday June 10, 2023 @03:15PM (#36273994)
Cloudflare Stream has beta support for webrtc ingestion and playback. If you did more research you may find other services or projects for scaling webrtc streaming to handle many viewers and handling transcoding to lower quality for people without enough bandwidth.

>And what is the proper infrastructure to scale WebRTC to thousands of listeners at once?

It looks like a pool of ingestion servers and a CDN for viewers. Ingestion servers handle getting the stream from the streamer and encoding it into whatever formats are needed. The CDN handles distributing this stream to people who want to watch it.

Re:OBS Merges WebRTC Support by jeroenhd on Saturday June 10, 2023 @03:10PM (#36273922)
Practically: however many your computer and uplink can serve. OBS needs to be tested, but this browser to browser test may give some indication: https://tensorworks.com.au/blog/webrtc-stream-limits-investi....

If you've got 100 viewers and can manage to saturate a gigabit uplink (which can be difficult because of routing and other upstream crap) you should be able to send about 9mbps video + overhead, which is pretty close to Twitch's bandwidth cap.

Because consumer ISPs generally don't have great routes between them and sustained gigabit uploads probably don't always work reliably because of overbooking with consumer lines, you'll probably be better off setting up an SFU on a cheap (temporary) cloud server somewhere.

Theoretically: media streams are identified by a 32 bit number, so about 4 billion clients.

Data streams are limited to 65535 endpoints (a signed 16 bit number with a reserved 0).

Re:OBS Merges WebRTC Support by martinald on Saturday June 10, 2023 @03:34PM (#36274201)
Because consumer ISPs generally don't have great routes between them and sustained gigabit uploads probably don't always work reliably because of overbooking with consumer lines, you'll probably be better off setting up an SFU on a cheap (temporary) cloud server somewhere.

Not sure why you think that? Maybe on cable modems, because DOCSIS is highly asymmetrical, but on FTTH GPON et al (pretty much the only technology which supports gigabit upload; docsis gigabit upload is extremely rare), there is no reason it couldn't saturate upstream at 1gigabit+. If anything, consumer ISPs are usually way more contended on the downlink side than the upstream side.

Re:OBS Merges WebRTC Support by bluefirebrand on Saturday June 10, 2023 @04:06PM (#36274524)
> Maybe on cable modems

Maybe I'm wrong but I'm pretty sure cable modems are still wha the vast majority of consumer households have.

Re:OBS Merges WebRTC Support by martinald on Saturday June 10, 2023 @05:39PM (#36275427)
Yes but the parent is referring to gigabit upstream connections. Very few gigabit upstream services are over DOCSIS. I don't know of a single provider that offers gigabit upstream on docsis. In the UK Virgin Media offer 100mbit/sec up on their top tier (1gigabit down). I think the fastest Comcast goes in the US is 200mbit/sec up over docsis and that's only in certain areas.
Re:OBS Merges WebRTC Support by jrockway on Saturday June 10, 2023 @04:26PM (#36274736)
GPON is typically shared bandwidth; 2.5Gbps for n customers (that could be an entire building, or the 4 houses adjacent to a box in the backyard, etc.)

More advanced fiber networks split bandwidth based on frequency ("color") instead of giving each ONT a time slot based on the current network capacity. But basically, if you're uploading at 1Gbps, that is fine until 1.5 more neighbors decide to do the same thing. It's rare. When I worked at Google Fiber we kept an eye on PON utilization and there was never a case where a customer couldn't get the requested bandwidth. We even had customers that maxed out 1Gbps/1Gbps 24/7 in an attempt to get banned or something, but of course we wouldn't ban you for that. We did worry about PON capacity and had some tech ready to go in case we ever needed to switch to truly dedicated per-customer gigabit connections.

At another ISP I worked at, our marketing material described the 1Gbps GPON-based link as "dedicated", but many customers noticed it was shared. It would slow down during peak business hours, and then get fast when other people in the building went home from work. We removed the term "dedicated" from our marketing. A political battle that I paid dearly for, but hey, they still exist and didn't get sued into oblivion by the FTC, so that's nice. (We also sold frequency-multiplexed 10Gbps circuits that really were dedicated... I suppose until our uplink to our ISP was saturated. I think we had 100Gbps transit and that never happened. But yeah, there is no such thing as dedicated unless you are physically plugged into the network that you want to reach. The Internet tries its best.)

Re:OBS Merges WebRTC Support by martinald on Saturday June 10, 2023 @05:37PM (#36275408)
I know that GPON is shared; but as you say it's rarely the 'weak link'. Regardless, even if it was, it's much more likely that the downstream would be saturated on the gpon level vs the upstream - consumer ISPs are much more downstream heavy than upstream heavy, so I have no idea what the parent post means when he says that its hard to saturated gigabit uplink on consumer connections. It's the opposite if anything.
Re:OBS Merges WebRTC Support by numpad0 on Saturday June 10, 2023 @05:49PM (#36275520)
It’s not about bandwidth inside the AS, but about fiber peering from ISP datacenters “to the Internet”. I don’t have anyone’s internal informations but that must cost 10-100x more than $35/Gbps/month. Residential internet is gym membership model, a best effort offering. Resources are always heavily over-committed based on typical use cases of groups of average users.
Re:OBS Merges WebRTC Support by jhugo on Saturday June 10, 2023 @08:27PM (#36276777)
“consumer ISPs generally don't have great routes between them” is the way more important part.

An eyeball network will generally have considerably more, and considerably higher bandwidth, peering relationships with content provider networks than with other eyeball networks.

At AVStack we experimented quite a bit with P2P multiparty video calls, it’s certainly possible but as you scale up the number of participants there’s huge variance in quality depending on the combination of ISPs used by the participants, the time of day, network conditions etc.

Re:OBS Merges WebRTC Support by weinzierl on Saturday June 10, 2023 @04:23PM (#36274705)
What do you mean by Twitch's bandwidth cap? Obviously Twitch supports more than 100 viewers, so is this an upstream cap from the streamer or a downstream per user cap or something different all together.

A glace over Twitch's broadcasting guidelines was not enlightening, but I'm clueless in these matters.

Re:OBS Merges WebRTC Support by ripdog on Saturday June 10, 2023 @05:12PM (#36275153)
He means the cap on the bitrate of the video that twitch will ingest. Twitch will then send this video to any number of viewers, as you say.
Re:OBS Merges WebRTC Support by jeroenhd on Monday June 12, 2023 @06:33PM (#36302255)
If you send more than about 6.75mbps of video to Twitch, you'll start getting errors. With the variable size of encoded streams you need to play with the cap a little to get a problem-free experience.
Re:OBS Merges WebRTC Support by felipellrocha on Saturday June 10, 2023 @02:57PM (#36273743)
> Sub-Second Latency - Create content and interact with viewers instantly. There is something magical about having a real conversation with your viewers.

How is this possible?

Re:OBS Merges WebRTC Support by Sean-Der on Saturday June 10, 2023 @03:14PM (#36273987)
It is pretty easy to get a one way trip time for packets that is sub-second! You see it with conferencing and other real-time communication things.

If you are curious on the 'how' of WebRTC I wrote a Free/Open Source book that goes into the details https://webrtcforthecurious.com/. Happy to answer any particular questions you have.

Re:OBS Merges WebRTC Support by evv on Saturday June 10, 2023 @03:39PM (#36274253)
You are a legend.
Re:OBS Merges WebRTC Support by beebeepka on Saturday June 10, 2023 @03:32PM (#36274174)
Bookmarked.

I added WebRTC support (audio, video, chat) to my already existing application a couple of years ago and felt the technology wasn't exactly ready for primetime. To me, it felt like the out of the box developer experience wasn't exactly great.

I am only saying this because I got to see exactly how much effort is required to get things going. Your work is greatly appreciated

Re:OBS Merges WebRTC Support by felipellrocha on Saturday June 10, 2023 @05:11PM (#36275141)
Man... I've been up all night, traveling all day, trying to adjust to the new timezone. Boy, I swear I read "sub-millisecond". Lol. Makes sense :) Thanks for the resource!
Re:OBS Merges WebRTC Support by numpad0 on Saturday June 10, 2023 @03:36PM (#36274218)
It just means the overhead is smaller for the feature set it offers. It’s not faster than bare MJPEG over UDP, but not way slower than that either.
Re:OBS Merges WebRTC Support by gruez on Saturday June 10, 2023 @03:01PM (#36273796)
Why shouldn't it be possible? Screen share on a Zoom/Teams meeting is basically a stream with sub-second latency.
Re:OBS Merges WebRTC Support by Karrot_Kream on Saturday June 10, 2023 @03:02PM (#36273808)
WebRTC attempts to set up a direct connection between the streamer and the receiver so packets are being sent over directly. This doesn't always work out, say if there's bad connectivity (e.g. NAT) somewhere so the connection uses a TURN server or if there's a conference call situation where an SFU is involved, but usually it works.
Re:OBS Merges WebRTC Support by andybak on Saturday June 10, 2023 @03:37PM (#36274233)
SFU?
Re:OBS Merges WebRTC Support by Karrot_Kream on Saturday June 10, 2023 @03:54PM (#36274414)
SFUs, or Selective Forwarding Units, receive media streams from multiple participants but only forward a subset of those streams to certain participants. For example, if you're in a Webinar of 20 people, instead of opening up direct connections to all 19 other participants (and having every other participant do so as well), you can open up a direct connection to the presenter and an SFU which will send you an aggregated, downsampled stream of all the other participants. It reduces direct connections across large calls and saves on processing power across all participants by aggregation.
Re:OBS Merges WebRTC Support by ShadowBanThis01 on Sunday June 11, 2023 @02:44PM (#36284334)
Then there’s STFU, which mutes all participants to save data on audio.
Re:OBS Merges WebRTC Support by xen2xen1 on Saturday June 10, 2023 @10:00PM (#36277210)
And easy to find for DOS or DDOS. Doesn't seem like a benifit.
Re:OBS Merges WebRTC Support by skykooler on Saturday June 10, 2023 @10:15PM (#36277291)
I ran into a situation yesterday where it doesn’t even work between two devices on the same wifi network. It very much depends on the particulars of your connection.
Re:OBS Merges WebRTC Support by hsudhduhsh on Saturday June 10, 2023 @03:02PM (#36273803)
most connections today are subsecond... a second is a lot of time.

also this is probably only taking California as the whole world, like everyone does.

Re:OBS Merges WebRTC Support by imtringued on Saturday June 10, 2023 @04:10PM (#36274573)
I have less latency to California than most Bluetooth headsets have to your ears.
Re:OBS Merges WebRTC Support by sitkack on Saturday June 10, 2023 @03:19PM (#36274047)
Can this do web RTC between servers? Can we construct a tree of OBS instances?

The numbers are purely illustrative. I don’t know what OPS can actually do but let’s say each OBS server can handle 50 connections that means if we had a two level deep tree, we could handle 2500 participants in a single call.

Re:OBS Merges WebRTC Support by Sean-Der on Saturday June 10, 2023 @03:24PM (#36274097)
I don't have an idea on how to do cascading OBS instances. I should write some software to make this easy though. I want to make 'self hosting' as easy as possible. I think https://github.com/glimesh/broadcast-box will fill that gap for now.

For my own usage I run WebRTC servers and have them 'replicate' the video traffic. LiveKit did a write up on this with some good graphics https://blog.livekit.io/scaling-webrtc-with-distributed-mesh...

Re:OBS Merges WebRTC Support by prox on Saturday June 10, 2023 @03:43PM (#36274291)
What I would like to see is a Twitch like experience but within your own space, aka what a website is for the web, and the hoster is nothing more than a facilitator. So you pay for the basic tech and maybe bandwidth but the rest is up to the streamer. Would be great to have a real alternative to Twitch that’s completely your own. People could still federate, but there is also a mainhub (broadcastbox.com or whatever)

Anyway it’s super exciting stuff! Great work!

Re:OBS Merges WebRTC Support by bluefirebrand on Saturday June 10, 2023 @04:04PM (#36274511)
I have been thinking about this a bit, kind of like a "what if you could set up a streaming website as easily as a wordpress blog"
Re:OBS Merges WebRTC Support by Sean-Der on Saturday June 10, 2023 @04:18PM (#36274658)
That was my hope for Broadcast Box! Mind checking it out and telling me what you think?

It has a docker image so you should be able to run a 'streaming site' with just a few commands :)

Re:OBS Merges WebRTC Support by prox on Saturday June 10, 2023 @05:07PM (#36275113)
What if you create extra facilities on a unified website? I would pay for that. Maybe sub functions, chat functions and so on. You need that for more discoverability I think.
Re:OBS Merges WebRTC Support by sitkack on Saturday June 10, 2023 @03:58PM (#36274459)
Thanks for the response, I’ll take a look. I really appreciate how much focus you put into this project. I think it does good for the world.
Re:OBS Merges WebRTC Support by pavlov on Saturday June 10, 2023 @03:23PM (#36274073)
This is output only (more specifically over WHIP, WebRTC-HTTP Ingest Protocol).

For incoming connections and compositing them you need something beyond OBS. Gstreamer is one option.

Re:OBS Merges WebRTC Support by viraptor on Saturday June 10, 2023 @03:35PM (#36274211)
> Serverless Streaming - WebRTC is P2P

As I understand webrtc, it describes the connection wrapping and the streams / negotiation, but not the discovery part. Which means it's about as p2p as http - if you add a way to find the endpoint and open sockets, you can make peers talk.

Is there something more in the OBS implementation that makes it more p2p? Am I missing something?

Re:OBS Merges WebRTC Support by notatoad on Saturday June 10, 2023 @03:45PM (#36274309)
i think the point is that the streams themselves require a lot of bandwidth, so you can make the expensive part of the process peer-to-peer instead of having to route the whole thing through a server and pay both inbound and outbound bandwidth costs on the stream.

it's not really meant to be a useful tool for making a pure peer-to-peer video service.

Re:OBS Merges WebRTC Support by viraptor on Saturday June 10, 2023 @03:56PM (#36274432)
Ok, so it's more of a "direct connection" than "peer to peer" scenario. That makes sense.
Re:OBS Merges WebRTC Support by jraph on Saturday June 10, 2023 @05:06PM (#36275098)
Doesn't P2P always require some kind of discovery mechanism unless peers know each others? For example for P2P file sharing with Torrent files, a central server (a tracker) that list the current peers? What is different between this and WebRTC? It's only partially decentralized but we still call this P2P, in both cases.
Re:OBS Merges WebRTC Support by Sean-Der on Saturday June 10, 2023 @05:30PM (#36275331)
Yes WebRTC does require a discovery mechanism. That is where WHIP/Signaling comes into play.

I have a chapter on signaling here https://webrtcforthecurious.com/docs/02-signaling/

I have an example of WebRTC without signaling https://github.com/pion/offline-browser-communication. Each side has to agree on things ahead of time. Useful for security cameras/IoT/LAN stuff though!

Re:OBS Merges WebRTC Support by eyegor on Saturday June 10, 2023 @06:46PM (#36276027)
Since you seem familiar with it, do you think it's possible to run a webrtc connection over http in a modern browser? I tried for a while to make this work but I gave up because it seemed like in js land it tries to use the getUserMedia apis which aren't allowed in an insecure context. The reason why I was trying was to stream audio + data locally from a device that emits its own wifi network (fully offline due to environment restrictions). It seems dumb to have to serve invalid self signed certs just to set up a webrtc ice handshake.
Re:OBS Merges WebRTC Support by Sean-Der on Saturday June 10, 2023 @07:15PM (#36276287)
Unfortunately no.

`localhost` is allowed. Could you reverse port forward the service to localhost? That will get around those restrictions.

Re:OBS Merges WebRTC Support by eyegor on Saturday June 10, 2023 @07:28PM (#36276392)
The service/webserver is running on the device, and clients connect via let's say a cell phone to the devices wifi network. The client talks to the webserver via a wifi network from the device. I don't think there's any way to trick the client into thinking the device is localhost? Maybe some HSTS shenanigans?
Re:OBS Merges WebRTC Support by jhugo on Saturday June 10, 2023 @08:06PM (#36276643)
Self signed certs would be much easier. Or a CA and certs signed by that; since it’s a controlled environment maybe you can install that CA to the trust store on the viewing devices.
Re:OBS Merges WebRTC Support by eyegor on Saturday June 10, 2023 @10:38PM (#36277446)
Unfortunately I don't control all of the clients and I'm using self signed certs now. It's just a hassle having to explain to non cs folks how to click past the unsafe prompt which looks to them like an error.

Meanwhile I can hack together a poor man's webrtc by sending raw packets over websockets and playback works just fine. I just lose all the real benefits of webrtc (robustness, udp vs tcp, etc.). It feels like it should be possible to work around since I don't access anything on the clients, they just play back a stream from the devices.

Re:OBS Merges WebRTC Support by jhugo on Sunday June 11, 2023 @12:52AM (#36278205)
If you’re serving DNS to the clients too, and you’re able to push updates to these devices regularly enough somehow, you can also use real certs with names under some public DNS zone that you control. (Make it resolve to a public IP on the Internet so you can issue/renew the cert, and a private IP on the private network.)

But yeah, one way or another, you need a secure context for WebRTC to work. That’s in the spec, and browsers follow it.

Perhaps in your context it really is nonsense, but how is the browser to know that your network/physical layer is secure?

Re:OBS Merges WebRTC Support by linuxdude314 on Sunday June 11, 2023 @02:03AM (#36278630)
What prevents you from using an actual cert from a trusted CA?
Re:OBS Merges WebRTC Support by the8472 on Sunday June 11, 2023 @03:17AM (#36279021)
> Doesn't P2P always require some kind of discovery mechanism unless peers know each others?

That discovery mechanism can be a decentralized network where any contact is an entrypoint. You connect to the network ~once and from that point onward everything is p2p and no fixed servers are involved or in control.

Re:OBS Merges WebRTC Support by capableweb on Saturday June 10, 2023 @09:11PM (#36276974)
"Peer to peer" is "direct connect".

Usually you'd divide what you're talking about into two different steps. Peer discovery, and peer transport.

Discovery is how you find other peers, sometimes its via DHT, sometimes it's hard-coded, other times just random walk. But just like in WebRTC, (and in TCP), you need to get the address from somewhere out of band, so you can connect, usually something like a centralized directory. But the transport of data, is P2P, just like TCP. So once you've found the peer and connected to it, all data goes directly between you.

Doesn't make it less/more P2P than any other P2P system out there.

Re:OBS Merges WebRTC Support by viraptor on Saturday June 10, 2023 @09:43PM (#36277140)
I'm not sure I understand what you mean as p2p transport. What makes HTTP or IMAP not p2p in this description? (You find the peer out of band (DNS) and connect directly to the peer with the data.)
Re:OBS Merges WebRTC Support by notatoad on Saturday June 10, 2023 @10:37PM (#36277440)
i'm pretty sure the parent is incorrect in describing TCP as a P2P protocol. TCP (like HTTP or IMAP) has strongly defined client and server roles, the two ends of the connection do not act as peers. P2P should mean that neither end of the connection is taking a specific role, there's no end that could really be classed as server or client.
Re:OBS Merges WebRTC Support by the8472 on Sunday June 11, 2023 @03:19AM (#36279031)
But OBS is the server serving the stream to everyone. It's using WebRTC in a client-server fashion. There's nothing P2P about it.
Re:OBS Merges WebRTC Support by ehsankia on Saturday June 10, 2023 @05:19PM (#36275227)
Woah, so if I understand this correctly, I can just put whatever streamkey i want in OBS with https://b.siobud.com/api/whip as my server, then my friend puts the same key on BroadcastBox hosted website and that's it? That's amazing. I do a lot of 1:1 streaming, sometimes with discord, sometimes with Twitch, but this is much better.
Re:OBS Merges WebRTC Support by Sean-Der on Saturday June 10, 2023 @05:32PM (#36275352)
Yea exactly!

https://b.siobud.com might not be always available. So best to host yourself for anything important :)

Excited for you to use it. Send any ideas my way for improvement.

Re:OBS Merges WebRTC Support by eminence32 on Saturday June 10, 2023 @10:59PM (#36277560)
Thanks for all your work getting this PR merged!

I've been trying it out for a while now (streaming to a self-hosted broadcast-box), and almost everyone who views the stream says that the video will freeze for 1 to 2 seconds every now and again (every few minutes). Audio is uninterrupted. Any hints about how I can debug this?

Re:OBS Merges WebRTC Support by doctorpangloss on Sunday June 11, 2023 @01:46AM (#36278542)
In my opinion, moderate your expectations of non-libwebrtc implementations like the one used in this feature.

Here's some speculation: Most users will experience non-libwebrtc WebRTC bugs (like OBS users). 1% of users who experience bugs in OBS WebRTC will report enough telemetry to fix the bug. But 99% of all of OBS's userbase uses Chrome, Edge or Safari, and maybe 50% of those (more than enough) report telemetry. So if their WebRTC bug was hardware / network specific - the bugs that matter - libwebrtc will have observed it and by now, probably fixed it.

Despite telemetry from hundreds of millions of users every day, it still took years for libwebrtc to reach truly robust, flawless stability on all the platforms it deploys on.

It's a complex technology.

Re:OBS Merges WebRTC Support by Sean-Der on Sunday June 11, 2023 @10:22AM (#36281678)
Sorry about that!

chrome://webrtc-internals should tell you the exact reason. My guess is packet loss and then it recovers on the keyframe.

In the next day or two I am gonna open a PR to set NACK buffer properly. Right now it is too small :/

Sorry you are hitting issues. Will try my best to fix it up! Email me [email protected] and would love to make sure we get it working 100%. I bet other users are hitting the same issues and not sharing.

Re:OBS Merges WebRTC Support by eminence32 on Monday June 12, 2023 @06:24PM (#36302144)
Thanks for your hint about chrome://webrtc-internals -- I'll see what that tells me!
Re:OBS Merges WebRTC Support by wisdow on Monday June 19, 2023 @04:52PM (#36396300)
Been experiencing the same issue. I tried to figure what causes this but without success. If you have any idea i'm down to discuss about it.
Re:OBS Merges WebRTC Support by throwaway485 on Sunday June 11, 2023 @01:31AM (#36278446)
> * Multitrack Input - Upload your transcodes instead of generating then server side. Give viewers multiple video tracks to see action from all sides.

I've always wanted this. Instead of the streamer switching video inputs we could select from the viewer-side which perspective we want. I've also thought about things like NASCAR partnering with Valve/Steam to use their Valve Index for 360-degree views from each car on the track. I don't know why they're not marking VR to people who love NASCAR, it'd be such an odd and likely successful niche. It'd be cool to accept-in multiple video inputs and even patch them together in realtime on the viewer side (unless they're specifically disparate).

Re:OBS Merges WebRTC Support by gloosx on Sunday June 11, 2023 @02:29AM (#36278760)
As a dev who works with WebRTC a lot, debunking some of these claims

* Serverless Streaming - WebRTC is P2P so you can video right into your browser. You don’t have to stand up a server anymore to stream for a small audience.

> Only possible in an ideal world, where NAT doesn't exist, in reality, you need to traverse NAT and it is NOT a serverless process.

* Sub-Second Latency - Create content and interact with viewers instantly. There is something magical about having a real conversation with your viewers.

> Only possible in an ideal world, where machines are very close to each other. In reality, you need expensive world-wide TURN clusters on top-notch infrastructure to ensure <100ms latencies you want.

* Mobility - WebRTC lets you switch networks at any time. Go from WiFi -> Mobile with zero interruptions.

> In fact, an interruption is happening when network conditions change – since you need to negotiate connectivity again. It is not seamless.

Re:OBS Merges WebRTC Support by fulafel on Sunday June 11, 2023 @03:22AM (#36279054)
The app data only need to go via a server if you don't have IPv6 and your IPv4 NAT is in the minority of NAT boxes that breaks the usual P2P NAT workarounds[1] like UDP hole punching. (Firewalls can of course also be configured to block it but that's another kettle of fish)

[1] https://www.cisco.com/c/en/us/support/docs/ip/network-addres...

Re:OBS Merges WebRTC Support by drdaeman on Sunday June 11, 2023 @04:04AM (#36279283)
I haven't bothered to learn why, but if both broadcaster and viewer are behind a symmetric NAT, usual STUN tricks don't always seem to work in practice. At least they haven't worked for me.

I have a DIY WebRTC forwarder for a RTSP stream, that lives on a home server behind a NAT. It has NAT because it lives in a cgroup that's isolated to IoT VLAN, and I haven't originally planned on WebRTC there, hoping I could make it work with restreaming over HTTP or Websockets. The NAT there is of the most common type: nftables' masquerade statement for the appropriate output interface, the usual conntrack allow established,related rule, and rule that allows outbound connections. For whatever reason, WebRTC only worked for me when my viewing device wasn't behind any other NATs.

Now, this is not a proper argument. Being a very lazy ass I haven't bothered with diagnostics so I don't know why it hadn't worked. I was already quite frustrated with various cross-browser JS shenanigans and after checking that STUN was indeed configured but had not worked, I've just punched a couple of port forwards and called it a day.

For whatever reason P2P streaming seem to work somewhat worse in practice than it should've been in theory. Usual computer leprechauns, I guess.

Re:OBS Merges WebRTC Support by Sean-Der on Sunday June 11, 2023 @09:56AM (#36281411)
You can traverse NAT with PCP/NAT-PMP. Libdatachannel doesn’t support it, but I don’t see why it shouldn’t in the future!

——

Yes <100ms is hard, I don’t think most users are looking for that. 400-600ms is what people can expect and I always see that!

—-

You don’t need to negotiate again with NICER

Re:OBS Merges WebRTC Support by gloosx on Monday June 12, 2023 @02:12AM (#36289050)
UPnP could be enabled on your NAT, but probably not, because it comes with certain security considerations. If you take most used networks globally – which are the cellular carrier networks, they won't let you do it. The point is about these techniques – you can't fully eliminate guesswork about whether specific NAT will be traversed or not for arbitrary peers, so the relay is mandatory to do it. If you're fine with only some part of the audience getting connected to your stream, and some not – you can go full P2P, otherwise the server-less claim is too bold.

400-600 ms is certainly not a "real conversation" experience, yet getting close to it. Speaking more realistically, the latencies spread would be much more broad depending on the peers' geography in a true P2P mesh.

If you don't need to negotiate again, how do you know which new socket to send bytes to? Is it some kind of a wizard, this NICER?

Re:OBS Merges WebRTC Support by erlend_sh on Sunday June 11, 2023 @02:30AM (#36278765)
> You don’t have to stand up a server anymore to stream for a small audience.

Now this is actually HUGE. It means Twitch has far less power as a gateway for greenfield streamers.

Re:OBS Merges WebRTC Support by starttoaster on Tuesday June 13, 2023 @01:49PM (#36313887)
Disagree. There is far more to Twitch than just "video goes here." There's the chat system for one, which this doesn't solve for. An important part of streaming is the social aspect of it to many, in the chatroom. It's also a way for your fans to communicate back to you. Sure they can open up your stream in a browser and chat with you in a Discord, but you're talking about requiring 2 tools being open as opposed to one. And that's not even the worst part: the subscription/follower system. The goal for any greenfield streamer is to gain a follower base. These followers and concurrent viewers are what allows you to get a "subscription" button for people to pay you. Otherwise you're asking for donations via Patreon or something similar? However, if you're getting your subscription money via Patreon, you're missing out on potential boat loads of money from Twitch Prime subscribers, where every Twitch user that also has an Amazon Prime subscription can send you one free subscription per month; a LOT of Twitch streamers make a lot of additional money this way, at no additional cost to the viewer.

Things you're giving up as a greenfield streamer by rolling their own streaming platform:

* The beginning of a follower base on your destination platform (Twitch)

* A "single pane of glass" for viewing your stream, chatting with you and other viewers, and giving you subscription/donation money. This is a deal breaker for a non-negligible number of people, and your follower base might not grow as large or as quickly as you'd like.

* Twitch Prime subscription money.