1. The memory loads are not stride 1. They're stride 12, effectively.
2. You have i8 -> i32 -> i8 operations going on. Mixed size conversion is generally problematic for vectorization.
3. You're storing 3 bytes of every 12 bytes. The compiler cannot generate extra stores, so it has to introduce masked stores. That's expensive.
The optimal vectorized code for this sort of algorithm is probably an unroll-and-jam: load 4 pixels, shuffle the vectors into r/g/b/a vectors for computation, do the computation, and then shuffle them back into r/g/b/a vectors for storage. This requires some code rewriting--inserting a dummy dest.a = dest.a * 1.0; might convince the compiler to store, although it could well eliminate the store before vectorization.
This sort of thing is why you end up needing a more specialized language for vector codes, such as ISPC.
Stride 16, no? Anyhow, that's a bug. I shouldn't have put the +=4 in the loop iteration increment. Once I fixed that and added a dummy set of dest.a, then I get auto-vectorization. Woo-hoo. See the link I posted in my reply to tomsmeding.
$ g++ -march=haswell -Ofast -fopt-info-vec-missed x.C -c
x.C:8:23: note: interleaved store with gaps
x.C:8:23: note: not vectorized: complicated access pattern.
x.C:8:23: note: bad data access.
x.C:8:23: note: not vectorized: not enough data-refs in basic block.
x.C:12:41: note: not vectorized: no grouped stores in basic block.
x.C:8:23: note: Build SLP failed: unrolling required in basic block SLP
x.C:8:23: note: Build SLP failed: unrolling required in basic block SLP
x.C:14:3: note: not vectorized: not enough data-refs in basic block.
and here's GCC nearly-9: $ g++ -march=haswell -fopt-info-vec-all -Ofast x.C -c
x.C:8:23: optimized: loop vectorized using 32 byte vectors
x.C:5:8: note: vectorized 1 loops in function.
x.C:12:14: note: ***** Re-trying analysis with vector size 16
x.C:8:23: note: ***** Re-trying analysis with vector size 16
In addition to what others have said, you typically need "restrict" on array args, and -ffast-math, or at least -funsafe-math-optimizations (in -Ofast), for things like reductions. (You need -funsafe-math-optimizations for NEON generally, per GCC docs.) A BLAS-like library still passes its tests with -ffast-math.I've recently had GCC 8 get about 2/3 the performance on haswell for a generic C optimized DGEMM micro-kernel as the hand-optimized one of similar structure which uses intrinsics/assembler. Without checking, the expert expects the performance difference to be due to prefetch and unrolling, which the generic kernel doesn't leave to the compiler.
As I understand it, -ffast-math is NOT useful here. That only effects floats and doubles.
I'm no black magician, but I think the fact that the operations you want to do in parallel (the r's of four adjacent pixels) do not access adjacent memory cells, requiring a gather instruction or somesuch, which I hear is either slow or unsupported. Alternatively, it could vectorise the pixels themselves, but that's three elements, not four. Does it auto-vectorise it if you also do the operation on the a's?
Good point. Combined with fixing my loop increment (that jcranmer made me realize was wrong), I now get some vectory goodness: https://godbolt.org/z/yTJzKH
No idea what it all means. It'll be interesting to benchmark.
dest.r = uint16_t(dest.r * invAlpha + c.r * c.a) >> 8;
Seems to get all of the benefit. The result is 34 instructions in the loop. (Was 38).Amazingly, switching from gcc 8.2 to gcc trunk, reduces the instructions in the loop to 18. See https://godbolt.org/z/sEvz2w
For example, here's a hello-world type SIMD problem: auto-vectorize a simple alpha blending pixel painting routine.
And here is GCC 8 _not_ auto-vectorizing it: https://godbolt.org/z/zPZL-JIs this the kind of thing you think should be auto-vectorizable by the compiler? If so, any idea how to persuade the compiler to do it?