Looking at the linked patches, you’ll note that the baseline (ff_detect_range_c) [1] is bog-standard scalar C code while the speedup is achieved in the AVX-512 version (ff_detect_rangeb_avx512) [2] of the same computation. FFmpeg devs prefer to write straight assembly using a library of vector-width-agnostic macros they maintain, but at a glance the equivalent code looks to be straightforwardly expressible in C with intrinsics if that’s more your jam. (Granted, that’s essentially assembly except with a register allocator, so the practical difference is limited.) The vectorization is most of the speedup, not the assembly.

To a first approximation, modern compilers can’t vectorize loops beyond the most trivial (say a dot product), and even that you’ll have to ask for (e.g. gcc -O3, which in other cases is often slower than -O2). So for mathy code like this they can easily be a couple dozen times behind in performance on wide vectors, especially when individual elements are small (like the 8-bit ones here).

Very tight scalar code, on modern superscalar CPUs... You can outcode a compiler by a meaningful margin, sometimes (my current example is a 40% speedup). But you have to be extremely careful (think dependency chains and execution port counts), and the opportunity does not come often (why are you writing scalar code anyway?..).

[1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346725.h...

[2] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346726.h...

Moreover the baseline _c function is compiled with -march=generic and -fno-tree-vectorize on GCC. Hence it's the best case comparison for handcrafted AVX512 code. And while it's is obviously faster and that's very cool, boasting the 100x may be misinterpreted by outsider readers.

I was commenting there with some suggested change and you can find more performance comparison [0].

For example with small adjustment to C and compiling it for AVX512:

  after (gcc -ftree-vectorize --march=znver4)
  detect_range_8_c:                                      285.6 ( 1.00x)
  detect_range_8_avx2:                                   256.0 ( 1.12x)
  detect_range_8_avx512:                                 107.6 ( 2.65x)

Also I argued that it may be a little bit misleading to post comparison without stating the compiler and flags used for said comparison [1].

P.S. There is related work to enable -ftree-vectorize by default [2]

[0] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346813.h...

[1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346794.h...

[2] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346439.h...

I think this comment should be on the top lol.

I mean, I love ffmpeg, I use it a lot and it's fantastic for my needs, but I've found their public persona often misleading and well, this just confirms my bias.

    > We made a 100x improvement over incredibly unoptimized C by writing heavily specific cpu instructions that the compiler cannot use because we don't allow it!

2x is still an improvement, but way less outstanding as they want it to publicize it because they used assembly.

> the equivalent code looks to be straightforwardly expressible in C with Intel intrinsics if that’s more your jam. (Granted, that’s essentially assembly except with a register allocator, so the practical difference is limited.) The vectorization is most of the speedup, not the assembly.

At my day job I have a small pile of code I'm responsible for which is a giant pile of intrinsics. We compile to GCC and MSVC. We have one function that is just a straight function. There are no loops, there is one branch. There is nothing that isn't either a vector intrinsic or an address calculation which I'm pretty sure is simple enough that it can be included in x86's fancy inline memory address calculation thingie. There's basically nothing for the compiler to do except translate vector intrinsics and address calculation from C into assembly.

The code when run in GCC is approximately twice as fast as MSVC.

The compiler is also responsible for register allocation, and MSVC is criminally insane at it.

One of these days I'll have to rewrite this function in assembly, just to get MSVC up to GCC speeds, but frankly I don't want to.

I’m not sure it’s register allocation. VC++ is indeed less than ideal, but 2x performance difference is IMO too much to explain by register allocation alone.

First check that you’re passing correct flags to VC++ compiler and linker: optimized release configuration, correct /arch switch, and ideally LTCG. BTW if you’re using cmake build system it’s rather hard to do, cmake support of VC++ compiler is not great.

Another thing, VC++ doesn’t like emitting giant functions. A possible reason for 2x difference VC++ failed to inline stuff, and your function has calls to other functions instead of a single large one. Note the compiler supports __forceinline keyword to work around that.

Compilers are extremely good considering the amount of crap they have to churn through but they have zero information (by default) about how the program is going to be used so it's not hard to beat them.

If anyone is curious to learn more, look up "profile-guided optimization" which observes the running program and feeds that information back into the compiler

It's AVX512 that makes the gains, not assembly. This kernel is simple enough that it wouldn't be measurably faster than C with AVX512 intrinsics.

And it's 100x because a) min/max have single instructions in SIMD vs cmp+cmov in scalar and b) it's operating in u8 precision so each AVX512 instruction does 64x min/max. So unlike the unoptimized scalar that has a throughput under 1 byte per cycle, the AVX512 version can saturate L1 and L2 bandwidth. (128B and 64B per cycle on Zen 5.)

But, this kernel is operating on an entire frame; if you have to go to L3 because it's more than a megapixel then the gain should halve (depending on CPU, but assuming Zen 5), and the gain decreases even more if the frame isn't resident in L3.

The AVX2 version was still 64x faster than the C one, so AVX-512 is just 50% improvement over that. Hand vectorized assembly is very much the key to the gains.

The only material difference AVX2 makes is that it can't saturate L1 bandwidth. Which would imply that 100x for AVX-512 is only for frames that fit within L1.

And... yep, the benchmark on 256x16 frames. [1]

[1] https://ffmpeg.org/pipermail/ffmpeg-devel/2025-July/346729.h...

If you ever dabble more closely in low level optimization, you will find the first instance of the C compile having a brain fart within less than an hour.

Random example: https://stackoverflow.com/questions/71343461/how-does-gcc-no...

The code in question was called quadrillions of times, so this actually mattered.

Almost all performance critical pieces of c/c++ libraries (including things as seemingly mundane as strlen) use specialized hand written assembly. Compilers are good enough for most people most of the time, but that’s only because most people aren’t writing software that is worth optimizing to this level from a financial perspective.

It's extremely easy to beat the compiler by dropping down to SIMD intrinsics. I recently wrote a 4 part .. guide? ..

https://scallywag.software/vim/blog/simd-perlin-noise-i

It's SIMD specifically. SIMD is crazy fast and compilers are still not good at generating it.

There are other things too like using the carry bit directly with ADC instead of using "tricks" to check for overflow before/after it happens for example.