News for hackers. Stuff that matters.
Re:Video Streaming at Scale with Kubernetes and RabbitMQ by thomasjudge on Monday October 09, 2023 @05:07PM (#37825491)
OP mentions that "I would love to be a little mouse and peek at YouTube’s complete architecture to see how far we are from them." You can occasionally find posts -often linked here- from another player in streaming video which you might have heard of, discussing technical architecture. For example, this might be a little lower level that you may be interested in as it relates to kernel optimizations to jack bit throughput rates, but I dig this sort of thing -

https://www.youtube.com/watch?v=36qZYL5RlgY

Re:Video Streaming at Scale with Kubernetes and RabbitMQ by c7DJTLrn on Monday October 09, 2023 @08:37PM (#37827240)
I've heard somewhere that YouTube has their own transcoding hardware.
Re:Video Streaming at Scale with Kubernetes and RabbitMQ by EricBurnett on Monday October 09, 2023 @09:35PM (#37827626)
Indeed. https://blog.youtube/inside-youtube/new-era-video-infrastruc... , https://research.google/pubs/pub50300/ . (Search the paper title and you should be able to find the pdf itself elsewhere).

I'm not actually sure on balance how much transcode gets done in hardware vs software, since it's also very amenable to using batch compute that's otherwise idle. I'll guess that most or all live transcoding - streams, on-the-fly transcode into formats not pregenerated - are done in hardware, and transcoding new formats for the back catalog are probably done on a mixture of mechanisms where and when capacity is available. (Source: Googler, not on YouTube though.)