Re:LLaMA Now Goes Faster on CPUs
by marshallward on Monday April 01, 2024 @11:04PM
(#39901979)
I'm sure there's more to it, but just comparing the profile output shows aggressive use of prefetch and broadcast instructions.
[ Parent
| on HN ]
BLIS does that in their kernels. I've tried doing that but was never able to get something better than half as good as MKL. The BLIS technique of tiling across k also requires atomics or an array of locks to write output.