Because you were hitting the power envelope limits in the CPU in these cases too. You might not see the heat, but the CPU cannot carry the power required to keep that core at non-AVX speeds with these power-hungry blocks operated at full speed.
As I said, to add insult to the injury, Intel didn't share the exact details of its AVX implementations and frequency ranges it operates, either.
Ah, publicly sharing your findings is/was forbidden too.
Whatever control mechanism is used in the AMD Zen CPUs is better than Intel's, so they downclock only when the power consumption really increases and the clock frequency recovers when the power consumption decreases, so there is no penalty when using sporadically some 512-bit instructions, like in the Intel CPUs.
On Skylake-X there are more states... AVX-512 light and heavy as well.
https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html#...
https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre...
Our systems doesn’t feature a similar override, but we can adjust the thermal and power envelope of the processors and system in general.
When we get a new bunch of systems, I’ll look into it, but my hopes are not that high.
Maybe we’ll get AMD systems this time, who knows.
It's not the heat load that slows you down. If you are using them enough that you produce enough heat that you have to downclock, it's still a win because the instructions improved your throughput more than what you lost in clocks.
The problem with Intel's initial AVX-512 implementation was that they didn't clock down because of heat, they clocked down pre-emptively and substantially whenever the CPU executed even a single AVX-512 instruction, even if there was no added heat load, and stayed on the lower clocks for a long period. This worked fine any proper SIMD loads, but was crushing in any situation where there was just a handful of AVX-512 ops between long stretches, such as using an AVX-512 optimized version of some library function.