News for hackers. Stuff that matters.
Re:Mistral AI Launches New 8x22B MOE Model by jart on Wednesday April 10, 2024 @08:10AM (#39989807)
The code that launches my code (see ggml_compute_forward_mul_mat) comes after CLBLAST, Accelerate, and OpenBLAS. The latter take precedence. So if you're not seeing any speedup in enabling them, it's probably because tinyBLAS has reached terms of equality with the BLAS. It's obviously nowhere near as fast as cuBLAS, but maybe PCIE memory transfer overhead explains it.

There are no comments on this comment yet.