News for hackers. Stuff that matters.
Re:Optimizing matrix multiplication in C by rurban on Sunday August 28, 2016 @06:48PM (#12378908)
The latest scatter/gather vectorization tricks are missing. The SSE improvements are only minimal.

Maybe a very new compiler, like Polly or ICC can vectorize this automatically.

ICC has a special -qopt-matmul option. https://software.intel.com/en-us/node/524953

Re:Optimizing matrix multiplication in C by saretired on Monday August 29, 2016 @12:06AM (#12379953)
Intel's Math Kernel Library has optimized versions of BLAS, a sparse solver, etc. I'd like to see a comparison using ICC/MKL.
Re:Optimizing matrix multiplication in C by sevenless on Sunday August 28, 2016 @07:00PM (#12378949)
I'd like to see support for optimized multiplication on symmetric matrices. I don't think BLAS can take advantage of that, certainly scipy/numpy doesn't.
Re:Optimizing matrix multiplication in C by santaclaus on Sunday August 28, 2016 @07:22PM (#12379027)
> I don't think BLAS can take advantage of that

There is BLAS level 2 (f|d)symv for matrix-vector multiplies and BLAS level 3 (f|d)symm for matrix-matrix multiplies. Last time I benchmarked symv, it was slower than the general implementation by around 25%...

Re:Optimizing matrix multiplication in C by sevenless on Sunday August 28, 2016 @08:55PM (#12379334)
Any idea why it would be slower? I liked the idea of avoiding half the multiplications.
Re:Optimizing matrix multiplication in C by Someone on Monday August 29, 2016 @06:17AM (#12380996)
Disclaimer: I have very, very, very little experience using BLAS. The reasons I post this are:

- the original poster gave an unqualified speed difference, which cannot reasonably be the full story. They likely left out information such as a 'for my use case' clause.

- I was curious, too, but couldn't Google benchmarks.

Having said that, my guess would be that it is slower for small matrices (where algorithm overhead plays a role), but faster for larger ones (where speed probably is proportional to memory access speed times amount of data accessed). There's a similarity here with searching a sorted array. There, a linear search is faster than a binary search up to a surprisingly large N.

I wouldn't dare guess where the cut-off point lies, but it likely lies at a point above where a matrix row fills a cache line (below that, reading only a few entries of a row brings in an entire row, anyways). For a level 1 cache line of 64 bytes, for floats, that would be a 16x16 matrix.

Re:Optimizing matrix multiplication in C by jackmott on Sunday August 28, 2016 @07:20PM (#12379018)
gather instructions on current cpus are not so good (unless that has changed in recent months), not always a speed increase from it.