cuBLAS benchmarks pretty well :) but sure, I am not super familiar with the state of CPU BLAS frameworks so I will take your word for it.

The good BLAS libraries are MKL and BLIS nowadays, I’m pretty sure.

BLIS is C and assembly if I remember correctly.

MKL is a big library, Wikipedia lists the languages used as C, C++, Fortran, and I think it is also pretty well known that they’ve sprinkled some magic assembly in there.

I believe MKL is mostly C/C++, although true extent is not really knowable.

The reply about cuBLAS was tongue in cheek as it is a GPU BLAS library

Intel hired Mr. Goto a while ago, he wrote gotoBLAS (from which openBlas is derived, so this one dude is responsible for a ton of FLOPs).

The thing that made gotoBLAS good was the hand-tuned assembly kernels, so we can be reasonably sure that MKL has hand tuned assembly kernels at this point.

I think the tongue-in-cheek comment is actually really good. It is a reminder that BLAS is more like a linear algebra API than a particular library.

I think I group hand-tuned assembly kernel and C in my mind because of the syntax sugar that C provides.

That’s fair practically, but in the eternal battle between conventional HPC languages and C, we can’t give C an unmitigated win. :)

Only someone at Intel could tell us for sure, but my understanding is that MKL is a C/C++ interface to a rather bog-standard BLAS/LAPACK library written in Fortran, but with various inner loops replaced or rewritten at runtime with a selection of assembly-optimized x86 SIMD codes chosen based on the capabilities of the CPU.

I’m sure it depends on the subroutine.

Reference BLAS is of course Fortran but nobody would actually use it. Some other good BLAS are (in order of historical to cutting edge), ATLAS, OpenBLAS, and then BLIS. They all mostly use C and assembly.

BLIS makes a good BLAS. It almost keeps up with MKL, according to the nice plots on their GitHub.

https://github.com/flame/blis/blob/master/docs/Performance.m...

They use C mostly, because the magic is in the GEMM (and possibly TRSM) kernels, which you are going to write in assembly anyway.

Intel also released oneDNN open source, which has an awful lot of BLAS-like functionality. No Fortran in there.

MKL is older than all these libraries of course (maybe not ATLAS, I’m not sure actually), and has a lot of other functionality (sparse stuff, FFTs), so I’m sure it has more Fortran! But it would be atypical compared to the rest of the BLAS implementations, if, like, their DGEMM was Fortran.