Google released the paper [1] "Warehouse-scale video acceleration: co-design and deployment in the wild " a few days ago.

https://news.ycombinator.com/item?id=26925307

>It does reduce flexibility when new video formats get released though.

A new video codec is released and adopted once every ten years. So I dont think it would be a problem. Hardware encoding also trade compression quality for speed. And they are compensating it with slightly higher bitrate. It will also reduce their incentives for improving open source encoder.

Although I think even Netflix switched to BEAMR ( Cant really blame them though )

I think the next frontier, for both Audio and Video will be Codec designed with LiveStreaming / Low Latency in mind.

( Or pretty much everything computing, I wish we could focus on latency, from Hardware Input, Display, Network, Disk, etc. Apple is certainly moving in that direction without talking about it. )

I was talking with a Netflix product manager a few years ago and, IIUC, he said Netflix re-encodes their entire catalog monthly to take advantage of new encoder optimizations. Disk is cheap. They want to optimize network transfers to both save money and improve user experience.

Netflix has about 36,000 hours of content. On Youtube that much content is uploaded every 1 hour and 20 minutes! Or to put it another way, Netflix re-encodes every month what YouTube encodes in just over an hour.

Your numbers are off. Netflix has more than 36.000 hours of content.

I don't know the reliability of these sources, but according to one article[1], as of 1 year ago, US Netflix had ~5.8K titles, totaling 36K hours of content. Another more recent source[2] claims that worldwide, Netflix has ~15K titles, so assuming the same distribution of runtimes, it would be closer to 93K hours of content. So, while you're correct, it's still the same order of magnitude.

sources:

[1] https://www.whats-on-netflix.com/news/how-long-would-it-take...

[2] https://www.comparitech.com/blog/vpn-privacy/netflix-statist...

Care to cite a source?

Back-of-the-envelope calculation: a slacker suffering from insomnia watching 5 streams of Netflix at once × 20 hours/day × 365 days non-stop would otherwise watch the whole catalog in less than a year.

Convinced?

Not really.

If we unpack your calculation to slightly more typical human behaviour: Watching 1 stream × 10 hours/day × 365 days non-stop, one person would need 10 years to watch the whole catalog.

Doesn't seem unreasonable to me.

You didn't get the core idea of my calculation: to show that Netflix has times of magnitude more than just 36k hours of content.

FWIW, that conversation about Netflix re-encoding their catalog monthly was from about seven years ago! Their process has surely changed since then. :)

Also, IIUC, they re-encoded about once a month, but the re-encoding didn't necessarily take one month of compute time.

Next codec is likely to be AV1. How does it fare in these situations ?

But also this might be more about protocols, but AFAIK streaming protocols have mostly been abandoned to the favor of chunk downloads over HTTPS ?

Facebook does have hardware for video transcoding. [1] https://youtu.be/0yk4SPRisPA

Hardware encoders are not as good because they have worse algorithms in order to make them faster. I don’t think you could make an x265 ASIC.

Also, good video codec engineers can’t be hired for any predictable amount of money, and ones who speak Verilog barely exist.

This used to be true but modern NVIDIA cards beat x264 on both quality and energy cost.

Any links for this? Since my experience shows that Ampere encoding blocks still aren't close to x264 / slow preset when it comes to saving bandwith and delivering quality.

Even with power savings it was usually more economically efficient to run encodes on a large (12+ core) machine than to deal with limited amount of nvenc slots on GPUs.

I believe it, and of course it's going to be better on energy cost. Is that because it supports newer codecs though, or is it on H.264?

They do?! Even on the slow presets?

NVENC beats x264 medium. Not quite up to the level of the "slow" presets yet but you have to throw a huge amount of hardware at it to match them let alone beat them. Basically the number I came up with a few weeks ago from playing around with x264 settings on ffmpeg was between 6 and 12 cores to keep up with NVENC at 720p and 1080p, depending on framerate and the quality preset.

What sort of bitrate are we talking about? I haven't tried using NVENC for years and last time I check it was clearly missing many details that x264 tries to preserve.

NVENC is good at cleaning the noise and fast encoding. ( Or basically Game Streaming ). Which isn't something you want to do if you want to do movies encoding.

> but you have to throw a huge amount of hardware at it to match them let alone beat them.

Sure, and I think hardware encoders are great when you need speed, certainly for real-time video. But in other cases, well, the medium preset sucks. I always encode videos at `veryslow`, and there's just no way to get close to that with a gpu.

How come ? This looks like to be easily parallelised, so GPUs should wipe the floor with CPUs ?

Video encoding isn't GPU-parallelizable. It's a good fit for either CPU+SIMD or custom ASICs. It's just a kind of compression, which means it's based on unpredictable if-statements, which is just what GPUs don't do.

You can mass parallelize it by encoding different clips on different CPUs, this is more optimal because it has less communication overhead.

No idea, maybe it will change some day. But right now, if you want the best quality at the smallest filesize, nothing seems to come close to the best software encoders. Maybe x264 and x265 are just really good.

They're unusually good because 1. encoding research wasn't done on a business schedule 2. isn't done according to objective metrics (the main ones used, SSIM/PSNR, suck and you have to use human raters) 3. more varied and weirder testcases (some pirated movies, some video game screen recording, more anime).

There's a lot of other free video encoding tools, like avisynth plugins, that are just better than all professional tools. I'm not sure why this is, maybe customers aren't sophisticated enough.

What about AV1 ?

Hardware encoders for newer codecs tend to be simply less bit-efficient. They can be more limited in features. My desktop's GPU can handle quite a few 1080p H.264 streams in parallel. But it can't do average-bitrate-capped CRF, which I think is the preferred type of encoding for streaming services.

Weird, NVENC supports this, I've used it for streaming before.

AMD consumer identified

Like someone already pointed out this whole announcement could be in response to Google announcing this week it's new Argos chipset approach to transcoding

[1] https://arstechnica.com/gadgets/2021/04/youtube-is-now-build...

The problem is that most of the big tech thinks they are smarter than 1000's of codec engineers who worked on libx264 and they try to reinvent the wheel. They realize too late that all the money they threw into building their own stuff (or variants) do far worse in general case than the baseline libx264. They just don't want to admit it.

I don’t think it’s that they don’t want to admit it, it’s that the cost savings from something disruptive may make it worth it at scale. Or that somebody wants to risk their career on it haha and big tech has tons of cash to throw around.

BidTechEngineer: Ya, I know, but I need that wasted money.

Absolutely. In fact, I interviewed a guy last week that comes from very big tech and he basically himself admitted that they're basically wasting time and money chasing unknown "ideal" codec while almost always those projects hit dead end. He loves the money, I am sure, but he wants out anyway. :)

Who wouldn't eventually?

I've been to a project where, because the development and "hardening" budgets were separate, we knowingly pushed out buggy code so that we could take advantage of the latter.

Had I known that half of the job would be to game the system, I wouldn't have joined.

> half of the job would be to game the system, I wouldn't have joined.

Bureaucracy is inevitable once a certain number of people join a group.

"Had I known that half of the job would be to game the system"

You should be happy it's only half!

...and I need that promotion to Senior Staff.

I think libx265 is the baseline now BTW (correct me if I'm wrong).

I would be suprised if FB doesn't use hardware encoders. That said, hardware encoders are fast, but they don't match the quality of the software (CPU) encoding. Slower CPU encoding still provide the best quality.

I can’t speak for all of the encoding options. But I can tell you that Dolby Hybrik, one of the main competitors and encoding space uses relatively cheap ec2 instances.

Based on the speed I see from Telestream and Elemental. I think they’re software based as well

Youtube and Facebook have a much different encoding problem then say HBO/Netflix/Hulu...etc

YT & FB have a few popular videos, and a ginormous long-tail. That long-tail needs to be encoded, and cheaply as it might not get many views.

The TV/Movie VOD over IP industry stands to benefit optimizing the encoding to have the smallest filesize with highest quality. Spending considerable CPU cycles to find that best quality path.

For those like Netflix, why would you invest with ASICs. You likely don't have a huge infra bill (relatively) when it comes to encoding. And CPU vs ASICs, the ASICs mean you loose flexibility to really fine-tune the quality and use the latest codecs.

For those like YT/FB. You just gotta encode that long tail. Getting something cheap that's 80% the quality of CPU-based encoding is good enough for 99% of the content.

Elemental is a mix of software and hardware encoding. They utilize GPU’s for some portion on the encode.

I could see lots of reasons to do software encoding at a company like Facebook which are probably not mentioned in the article, including e.g. extracting the optical flow which can be reused as part of various machine learning workflows.

For example, if you want to only annotate 1 in 100 frames and interpolate with optical flow, you can get the flow in the process of doing video compression.

I don't have inside info but there are various other things like this that I can think of for wanting to do it in software.

Also, hardware encoders suck if the company making the hardware decides to suddenly EOL the product. With software encoders you can easily scale the backend whenever you want, at any time in the future, and without succumbing to supply chain issues.

If you’re Facebook, the company making the hardware doesn’t suddenly decide to EOL the product, or they are sued for millions for breach of contract.

I've seen this contrasted with the Google stuff but it seemed to me the two approaches were entirely complementary.

Facebook, which videos will we transcode into what versions.

Google, with this list of videos to transcode how can we do it cheaply on a massive scale.

I didn't spot anything that pitted approaches against each other.

counterpoint: if you're big enough to have your own bona-fide cloud, then you might have more spare CPU capacity that can be flexibly allocated than available PCs with dedicated GPU

If you are in the cloud business you are worried about heating power, with powering down disks a real thing

? Xilinx for the rescue?

What was mentioned in the article seem like very basic, obvious considerations. Meanwhile, Youtube has custom ASICs which efficiently produce multiple formats simultaneously.

And yet, despite YouTube's abundantly clear motivation for doing so it took them years and years to develop the hardware and it only recently hit production. Could it be that acceleration for video encoding isn't as easy as people are making it sound?

I'm waiting for the "meh, i could roll it out in a weekend" responses.

Of course I could;) It would just be hacky, ugly, unreliable, scale badly, use either GPUs or inadequate FPGAs, and fall over once it exceeded 10 users. My resume doesn't need to know that, though:)

You now know 1 way of not to do it.