Running on Rosetta (and looking like an x86 cpu without fma) is likely a contributing factor, but the M1 was much slower for sums and sums of squares than a Cascadelake (Intel, avx512) cpu here:

https://github.com/chriselrod/VectorizationBase.jl/issues/22

I do a lot of SIMD optimization, so it'll be interesting to see how the M1 competes once there's native support in Julia.