News for hackers. Stuff that matters.

taxemeEvasion

Re:Matrix multiplication inches closer to mythic goal on Tuesday March 23, 2021 @12:03PM (#26556607)
It's a main reason why Fortran still has such a hold on the SciComp/HPC community.
Re:Matrix multiplication inches closer to mythic goal on Tuesday March 23, 2021 @11:56AM (#26556520)
Strassen's is sometimes competitive in the thousands!

https://www.cs.utexas.edu/users/flame/pubs/FLAWN79.pdf

You're right though afaik all the other algorithms are still 'galactic' and completely impractical

Re:Matrix multiplication inches closer to mythic goal on Tuesday March 23, 2021 @11:51AM (#26556466)
I think its also important to mention that these results are for the multiplication of two general dense matrices and full floating point accuracy. If your matrix is structured (sparse, sparse in the Fourier domain, low rank, H, HSS, etc) you can usually exploit that structure to break up the multiplication and reduce the work complexity dramatically. (These rely on building blocks of the dense general matrix results)
Re:Matrix multiplication inches closer to mythic goal on Tuesday March 23, 2021 @11:43AM (#26556347)
It's typically still the O(N^3) method that's implemented in OpenBLAS, BLIS, cuBLAS, MKL, etc. There are some high performance implementations of Strassen's Algorithm with a fixed level of recursion that start to perform much better as the matrix size gets large (see the work from UT Austin's BLIS). From my understanding for the more complex algorithms the hidden constant simply grows too large to be worth it on conventional hardware.

As another poster commented, these are O(N^3) in work, not necessarily the wall clock time you see due to parallel speedup and cache effects, but will scale as O(N^3) once you're at a size past all of these. These implementations are highly tuned and optimized for hardware, much much more so than the naive 3 loop implementation. The predictable and 'easy' access patterns make the simple algorithm hard to beat.