Let that sink in:

'Encoding run time was 9226.4x, 8139.2x and 667.1x greater, respectively…'

The codec is good, but the encoder has a long way to go. This places it squarely in the realm of only worth it right now if you're serving the same video millions of times (someone graph this?).

It's a bit hyperbolic to claim that this is a 'practical use case', but maybe at Facebook/Netflix/YouTube's scale, it is. But it will be exciting to watch this space.

The same thing was true of H264 when it first came out. Reference encoders/decoders are always incredibly slow -- they're written to be correct, not fast.

> they're written to be correct, not fast.

AOM dev here. I'd love you to be right, but that's unfortunately not the case. They're written to improve VQ.

Regarding the "written to be correct" part of your sentence:

There's no emphasis on correctness. For monthes, aomenc has been accidentaly using different transforms sets for 8 bit and 10 bits. It also used to compute its RDO based on the legacy probability model (non-CDF) although the result was binarized using the new one (CDF)). Moreover, serious regressions (decoder can't decode the encoder's output, lossy lossless) are introduced almost every day (!), fixed the next day.

Now, regarding the "not fast" part of your sentence:

There's been a lot of work on SIMD (Current aomenc performance is measured with SIMD optimizations). Actually, the code features complex mechanisms (C header file generation from Perl scripts at configure time) to handle the dispatching between the optimizations. To make the matter worse, almost all the pixel prediction code is physically duplicated in two versions: the generic code that uses 16-bit pixels, and the 8-bit optimized code that uses 8-bit pixels. All of these are actually huge hindrances to performance-unrelated developments.

What most people don't realize is that diminishing returns would be inevitable even if there wasn't a theoretical limit to compression. While it's true that computers are getting more powerful all the time, we're also getting more bandwidth and storage space all the time too. Even if it were possible to achieve results on a visual par with x264 in half the encoding size, the computational complexity would be so overwhelmingly high that it's not hard to see why in most cases you'd want to stick with the faster codec simply because of encoding time and power use on consumer devices.

It really wouldn't be that surprising if we're still using video encoders with efficiencies somewhere in the h264-AOM range 1000 years from now. Cases where the tradeoffs make higher efficiency worthwhile are hard to come by.

Many continue to use this comparison.

H264 Reference Encoder wasn't even tuned for anything. Not even quality. Hence why we not only get much better speed in x264, but also higher quality.

And the Reference Encoder ( JM ) was only 250 times slower when it first came out. It truly was a pieces written to be correct rather then anything else. And there was some easy wins to get it 10x - 20x faster within a matter of weeks.

Now Av1 is nearly 10000x slower, that is if you manage to speed it up 100x it is still 100x slower. The codec right now was tuned for quality metrics. They were trying to hit PSNR, SSIM and even VRMF at 30% higher then VP9 at all cost. Which is not the same as JM.

I dont doubt they could speed it up. But by how much. Some even mention it is only usable by companies with Scale, well this is so far from truth. You dont gain much economy of scale once you past certain level of compute. What cost $1M dollar to encode at facebook now cost $1B or $10B as it stands. People seems to have a real lack of sense in scale of how much 1000x is. Or even 10K x. Your compute cost is going to be dictated by CPU manufacturer or silicon prices. And that 30% of bitrate reduction is not going to save you money from the insane increase of compute. Especially if you are Netflix with Appliance sitting at ISP's DC.

Not long ago I asked how are we going to get the encoding speed to some sane's time. Like 10x of x264 High Quality. Turns out it will very likely have to disable certain tools set now being used, which means a reduction of quality. But I believe it will still be better then x265 at the end. The tools available for AV1 is quite massive.

Right now we have xvc which is looking more like an H.266 encoder, and offer 30% improvement over AV1 if you want to believe PSNR and SSIM. But it is even slower then AV1.

It seems AV1 allows to rotate reference blocks in inter prediction. I wonder what an algorithm that would search for a prediction MV would look like (not even counting bidir).

Even without rotation, MV search can get sophisticated and not exactly intuitive in a typical encoder. And with HEVC having "merge" that win that much bits, it's not like there was so much bits left to earn with the predictor itself.

To get to “sane time”, you break up the video and have more than one encoder running at a time: http://cseweb.ucsd.edu/~gmporter/papers/nsdi17-excamera.pdf

That's only going to give you a 4x improvement on typical devices and is something HEVC can do too.

Why only 4x? Video can be subdivided into more pieces.

Because most devices are quad-core. After a certain point context-switches create a diminishing return.

Distribute across multiple machines.

and for consumers, as long as the decoder is fast and cheap...

I'd assume encoding was largely software-based using a POC encoder while x264/vp9 video encoders probably already take advantage of hardware acceleration shortcuts evolved over many years.

> the main focus of current AV1 development is on speed optimization to make it practical for use in production systems

Remember: first make it work, then make it right, then make it fast. Seems they're only starting the 3rd step now.

No, it wasn’t hardware against software.

But, and it’s hard to understate this, software like x264 has many man years of intricate and clever optimization work under its belt. Just look through some of the development history to see the kind of hoops jumped through and low level clock tick counting tweaks that have been made.

Also, I believe in general the expectations around AV1 have always been set realistically in the sense most are expecting that it will likely always require more compute to achieve a significant portion of any superior compression.

Important as those caveats may be, clearly there is a long way to go and you may still legitimately ask how do we know a 3 orders of magnitude deficit can be made practical?

In truth we don’t know with certainty where it will end up, that is true. However the bet was not made by guessing. There has been lots of investigation and analysis of what kind of implementations may be possible including design review with hardware engineers for that side of things.

So essentially it’s designed to outperform the current state of the art, and some high stakes educated projections have been made that it will become practical within a timeframe that still makes AV1 competitive enough to be a worthwhile value proposition.

libvpx, the VP9 encoder library used in this test, has no support for any hardware encoder blocks for VP9 [1], so it does everything in software. There are ways [2] to compile some support into ffmpeg-with-libvpx that makes it able to invoke the hardware encoder in newer Intel CPUs (Skylake or newer) [3][4] (using vp9_vaapi) but it's doubtful that this was used, since their command-line switches indicate nothing of the sort.

x264 is a software-only encoder that provides no hooks into hardware acceleration. Their ffmpeg command line indicates that no hardware acceleration hooks of ffmpeg were used.

[1] https://github.com/webmproject/libvpx/blob/master/CHANGELOG [2] https://gist.github.com/Brainiarc7/24de2edef08866c3040805048... [3] https://cgit.freedesktop.org/libva/commit/?id=fb57f5c15e72c3... [4] https://en.wikipedia.org/wiki/VP9#Hardware_implementations

but let's remember that x264 has had a lot of time to find drop optimizations in simd use and time for CPUs to start optimising for it as a use case.

x264 runs circles around all the hardware accelerated encoders, at least quality wise.

I haven't tried AV1 yet, but the only way I was able to encode with x265 in a reasonable amount of time was to shard the job over multiple machines and then join the file back together at the end.

x265 gets ~10fps on veryslow with 480p video on a recent workstation. Intel and AMD are pretty much tied there because Intel's per-core performance is significantly higher for x265, but AMD has more cores.

x264 is a software codec.

>The codec is good, but the encoder has a long way to go.

True, but it hasn't been optimized as of yet, I'm pretty sure there are a lot of low-hanging fruits once they start converting code hot paths into hand-optimized assembly.

That said, no matter how much optimization it gets, it will be slower than equally optimized HEVC, h264 given that AV1 is a more complex codec.

On the flip side, as the encoder gets better, the quality and bitrate will improve as well, not just the encoding time.

Netflix yes, others no, most videos aren't played often

Certainly some videos on youtube, and which ones should be pretty predictable.

It doesn't even need to be predictable, just reactive.

At least 2 new software encoders are coming by and they are way faster than this, while achieving the same (or better?) quality than libaom.

Do you have any more info on this?

Names?

Note that this is for the AV1 reference software encoder. It's not hardware accelerated, and probably built with limited regards to speed and e.g. vectorisation.

Much worse than I tough. With this complexity increase, HEVC is by far the best choice.

For now - and even HEVC encoders are still pretty new.