Salsify – A New Architecture for Real-time Internet Video

Re:Salsify – A New Architecture for Real-time Internet Video by Sean-Der on Sunday August 25, 2019 @07:20PM (#20796292)

Hey Keith,

Amazing work, I am really impressed with what you are doing. I have never found any good reading in this area beside your work, the only thing I have seen is GStreamer's rtpjitterbuffer and libwebrtc. I haven't really felt confident in what I have learned from either though.

Have you thought about testing/comparing against other WebRTC implementations? It makes me a little sad to see WebRTC get a bad name just because one implementation has issues. WebRTC is a huge opportunity to get companies to invest in one thing, instead of reinventing the wheel and locking people in. Do you think things could be improved and eventually match Salsify?

Somewhat unrelated but I am working on Pion WebRTC[0]. By design it is not coupled tightly with the encoder/decoder (but I want to give feedback to the user so it could influence it if they wanted). Do you have any suggested readings, would you be ok with me contacting you directly? I really want to build something elegant that allows people to build amazing things with WebRTC. Right now I have been having people do things manually, I don't want to ship something without doing my due diligence.

thanks

[0] https://github.com/pion/webrtc

[ Parent | on HN ]

Re:Salsify – A New Architecture for Real-time Internet Video by keithwinstein on Sunday August 25, 2019 @07:57PM (#20796478)

Thanks for your kind words -- you're certainly welcome to contact us if you think we can help. Yes, as I wrote elsewhere here, it would probably be possible to do everything Salsify does within the context of WebRTC, if you get to change the sender and receiver. If you're only changing the sender and you want to interoperate with receivers running the WebRTC.org codebase (e.g. Chrome), you have less flexibility. If you're only changing the sender and you want to work with existing high-performance video codecs and the API they expose, even less flexibility.

My main advice for implementers would be, benchmark your end-to-end glass-to-glass video latency (including the time spent waiting for the next frame to be captured & encoded, and then the end-to-end latency through to the display of that frame) over varied/unpredictable networks. In my experience, implementers sometimes get caught up focusing too much on network-layer measurements (IP latency) and can end up missing what in my view is the bottom line: glass-to-glass video latency. You can use our mahimahi mm-delay/mm-link/mm-onoff/mm-loss tools (part of Debian/Ubuntu) and the included traces to model some bad networks. And then see how you do on the same traces we use in our paper. If you can make a plot like our Figure 8(a) and it all looks good, that seems like good progress in my book.

[ Parent | on HN ]

Re:Salsify – A New Architecture for Real-time Internet Video by dtaht99 on Tuesday August 27, 2019 @04:15PM (#20813995)

Nice to see you keith. I agree that the network latency part of the equation tends to get too much play, where glass to glass is the better metric. On the other hand the bufferbloat effort started where network latencies were often the dominant contributor (seconds!) and for those that have actually implemented things like your e2e algos and/or installed stuff like fq_codel/fq_pie/sch_cake - the encoding step now dominates.

'Course, I still kind of miss scan-lines, and had hoped we'd finally all have enough bandwidth to just blast raw frames over the network by now.

/me hides

[ Parent | on HN ]