This concern really only applies to synthetic benchmarks (stuff like SPEC CPU). If you're testing a commercially available application or game as delivered to consumers, this issue does not invalidate the benchmark, it just makes the software vendor a bit of an Intel stooge.

Of course it doesn't matter for home consumers because real target audience for MKL isn't desktop user applications or games. Try using VASP or COMSOL or Mathematica on an AMD CPU. Both MKL and CUDA are major "issues" in HPC that limit decisions in purchasing clusters.

I don't know about VASP because it's proprietary, but I have compared other DFT code on 64-core Bulldozer with 12-core Sandybridge nodes. I don't remember the numbers, but the high core count was rather effective in reducing the communication costs with all free software.

https://www.archer2.ac.uk/ will run a lot of that sort of thing. I think at least cp2k and CASTEP are included in the benchmarks, but they're listed somewhere.

When you're actually a part of conversations on purchasing a cluster (which I gather from the way you talk, you haven't been a part of) which cost $400k ~ $1M for a smallish/mid-sized, arguments like "I don't remember the numbers" "they're listed somewhere", "there's this some other random DFT code" aren't effective. The hard fact is (which I hate), you get results faster with MKL on Intel than any other alternatives. This is more so with the proprietary software that are golden standards.

> I have compared other DFT code on 64-core Bulldozer with 12-core Sandybridge nodes

And what's that comparison supposed to tell us, aside from the obvious fact that MPI introduces latency? That's just about the number of cores, not the performance of a each core. You need to compare 64-core AMD node against a 64-core Intel node.

I don't remember, because the measurements on the £1M purchase was maybe five years ago, but they taught a useful lesson. I didn't see figures in what I was responding to. If I'd had more influence on the purchase, as opposed to observing the process, we wouldn't have ended up with a pure sandybridge system, which was a mistake. Anyhow, my all-free-software version of cp2k was faster on it than an all-Intel version on slightly faster CPUs on an otherwise equivalent cluster. I measured and paid attention to the MPI, which benefited everything using alltoallv. The large core-count AMD boxes were simply a better bet for the range of work on a university HPC system. It's not as if most codes topped out an arithmetic intensity and there was a serious problem with serial performance, even if MKL had been significantly better than the free libraries, which it wasn't.

For a recent exercise, spending rather more money on AMD CPUs for the UK Tier 1 system, look at the Archer2 reference and benchmarking for it. It's expected to run large amounts of VASP-like code; www.archer.ac.uk publishes usage of the current system. Circumstances differ, and I'm pointing out contrary experience, understanding the measurements and what determined them.

The go-to Intel-crushing benchmark these days is Cinebench R20, which runs on Intel's Embree raytracer.

So Cinebench R20 is a suspect?! That's not good...

But is that compiled by ICC?

Well, it depends. If you are a big enough outlet or a hyperscaler, you should care about raw-performance because if a sufficient amount of people make the same choice as you, the software vendor will adapt and you will end up with the competitive advantage.

Most of Python math libraries? So a majority of Machine Learning folks is affected.

Most Python math and ML libraries are compiled using GCC or LLVM and are linked against against cBLAS or OpenBLAS[1]. The latter is highly performant on both Intel and AMD (and other platforms). Some libraries are optionally compiled against the MKL, in particular those that are distributed with the Anaconda Python distribution.

[1] https://numpy.org/doc/stable/reference/routines.linalg.html?...

PyTorch uses MKL https://www.google.com/amp/s/amp.reddit.com/r/MachineLearnin... and the workaround for AMD has been disabled by Intel

I think MKL actually fixed Zen performance. That is, the workaround no longer makes any difference because it is no longer needed.

Small matrix multiply benchmarks on a Zen2 (Ryzen 7 4700U), featuring MKL 2020.1.216+0, OpenBLAS, and Eigen: https://gist.github.com/stillyslalom/bd916e3d26b4531364676ac...

MKL's benchmark performance requires AVX + FMA. 3.5 GHz * (4 add + 4 multiply) * 2 fma/cycle = 56 peak GFLOPS. To exceed 50 GFLOPS without them would imply the CPU ran at 12.5 GHz.

OpenBLAS, on the other hand, actually performed poorly because it was limited to SSE thanks to a bug preventing the CPU from being recognized as Zen2.

I think MKL actually fixed Zen performance. That is, the workaround no longer makes any difference because it is no longer needed.

Odd. I am trying on my 3700X and it is definitely not using AVX, FMA or AVX2 code paths. Intel MKL 2020 update 2:

     ldd  ~/git/sticker2/target/release/sticker2  | grep mkl_intel
     libmkl_intel_lp64.so => /nix/store/jpjwkkv1dqk4nn8swjzr5qqzp0dpzk2f-mkl-2020.2.254/lib/libmkl_intel_lp64.so (0x00007fe786862000)

I checked the instructions in with perf and it is using an SSE code path. Also, as reported elsewhere, MKL_DEBUG_CPU_TYPE=5 does not enable AVX2 support as it used to do.

Comparing OpenBLAS and MKL with `peakflops` in Julia, there's definitely an advantage for MKL:

    julia> using LinearAlgebra

    julia> BLAS.vendor()
    :openblas64

    julia> BLAS.set_num_threads(1)

    julia> peakflops()
    3.9023447970402664e10


    julia> using LinearAlgebra
    
    julia> BLAS.vendor()
    :mkl
    
    julia> BLAS.set_num_threads(1)
    
    julia> peakflops()
    4.8113846984735275e10

That's close to the ~50 Gflops I saw in @celrod's benchmarks.

The plot thickens. As I reported elsewhere in the thread, the slow code paths were selected on my machine, unless I override the mkl_serv_intel_cpu_true function to always return true. However, this was with PyTorch.

I have now also compiled the ACE DGEMM benchmark and linked against MKL iomp:

    $ ./mt-dgemm 1000 | grep GFLOP
    GFLOP/s rate:         69.124168 GF/s

Most-used function is

   mt-dgemm  libmkl_def.so       [.] mkl_blas_def_dgemm_kernel_zen

So, it is clearly using a GEMM kernel. Now I wonder what is different between PyTorch and this simple benchmark, causing PyTorch to result in a slow SSE code path.

Found the discrepancy. I use single precision in PyTorch. When I benchmark sgemm, the SSE code path is selected.

Conclusion: MKL detects Zen now, but currently only implements a Zen code path for dgemm and not for sgemm. To get good performance for sgemm, you have to fake being an Intel CPU.

Edit, longer description: https://github.com/pytorch/builder/issues/504

Hmm.

FWIW, on my [Skylake/Cascadelake]-X Intel systems, Intel's compilers performed well, almost always outperforming GCC and Clang. But on Zen, their performance was terrible. So I was happy to see that MKL, unlike the compilers, did not appear to gimp AMD.

It's disappointing that MKL doesn't use optimized code paths on the 3700X.

I messaged the person who actually ran the benchmarks and owns the laptop, asking them to chime in with more information. I'm just the person who wrote that benchmark suite.

It seems I have found the issue. We were both right. MKL now uses a Zen-optimized kernel for dgemm, but not (yet?) for sgemm. More details:

https://github.com/pytorch/builder/issues/504

If OpenBLAS' CPU detection fails, you can force it with an environment variable, but why omit AMD's implementation?

Do you know if this applies to epyc as well?

No, I don't. The 32-core AWS systems must be Epyc, so I'll try benchmarking there.

When OpenBLAS identifies the arch, it is competitive with MKL in single threaded performance, at least for matrices with a couple hundred rows and columns or more. But MKL truly shines with multiple threads, so scaling on a 32 core system would be interesting to look at.

You can see BLIS on Intel's home turf at https://github.com/flame/blis/blob/master/docs/Performance.m... (52-core SKX) and compare with OpenBLAS on 32-core Zen1. (Multithreaded BLAS isn't typically used in HPC, where the parallelism is elsewhere.)

I'd have to disagree, whenever I install new pip/conda packages, an MKL version is downloaded by default (I've never seen any *BLAS version by default in my life).

NumPy binary wheels on PyPI (i.e. from pip) are built with OpenBLAS. NumPy from the official Anaconda, Inc. conda channel defaults to MKL. The conda-forge channel defaults to OpenBLAS.

Explained in the NumPy docs here: https://numpy.org/install/

> Most Python math and ML libraries are compiled using GCC or LLVM and are linked against against cBLAS or OpenBLAS[1]

This is definitely understating MKLs market share in BLAS. It's an extremely common BLAS backing for Python libraries.

MKL is a real problem in HPC, and it does play a significant role in decisions when purchasing clusters.

In addition, software like Mathematica/Matlab which use only MKL can affect decisions even for office workstations.