News for hackers. Stuff that matters.
Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by flakiness on Friday July 26, 2024 @02:34PM (#41081040)
Note that this is from 2022.

My guess is that people nowadays are gradually moving away from raw CUDA programming and moving towards things like Triton etc, and you won't be focusing on pure GEMM since you tend to do some fusion.

The Triton tutorial claims their performance is on par with cuBLAS.

https://triton-lang.org/main/getting-started/tutorials/03-ma...

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by machinekob on Friday July 26, 2024 @03:15PM (#41081398)
But Triton is just abstraction over CUDA for Python (same like cupy, numba etc.). If you need low lvl access you still will use CUDA if you want high level you can use Triton or numba even higer you'll just use wrappers like pytorch/jax.
Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by imjonse on Friday July 26, 2024 @03:22PM (#41081453)
It is still an excellent article if you care about how the GPU works. Triton doesn't magically hide all the hardware details.
Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by dang on Friday July 26, 2024 @04:00PM (#41081749)
Year added above. Thanks!
Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by bee_rider on Friday July 26, 2024 @03:18PM (#41081416)
I’m sure they are using cool super-modern stuff but I wonder if Eigen can somehow be made to spit out CUDA code?
Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by lights0123 on Friday July 26, 2024 @10:22PM (#41084043)
Since 2016, you can use Eigen in CUDA C++ code and it just works: https://eigen.tuxfamily.org/dox/TopicCUDA.html
Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by bee_rider on Saturday July 27, 2024 @09:07AM (#41086335)
Oh wow, that is cool as heck.
Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by almostgotcaught on Friday July 26, 2024 @10:04PM (#41083970)
> My guess is that people nowadays are gradually moving away from raw CUDA programming and moving towards things like Triton etc

Your guess is wrong. Besides the fact that there's much more to life than matmul (for which triton is just ok), the other obvious fact is that triton has exactly 1 frontend (python) and there's much more to life than that frontend.

I find that basically in every thread about low-level work there's someone making some weird comment about how triton or mojo or XYZ supplants CUDA or assembly or whatever. I can't understand how this comes about because absolutely no one working in these areas thinks XYZ is going to supplant anything. So it's invariably outsiders making these claims and I cannot fathom why any outsider would be motivated to make claims from the outside.

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by the_svd_doctor on Friday July 26, 2024 @11:57PM (#41084345)
FWIW Triton can actually be used pretty easily from C++ straight using MLIR. XLA does that. Maybe not exactly a frontend but not that far.
Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by almostgotcaught on Saturday July 27, 2024 @12:16AM (#41084425)
> FWIW Triton

i have PRs in Triton - i'm well familiar with the fact that triton is an MLIR project.

> C++ straight using MLIR

that's like saying llvm ir is usable through C++ ... or hell that's like saying NVPTX is usable through C++. it's not just not a frontend it's the exact opposite: it's emitting IR using IR builders i.e. writing a compiler.

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by flakiness on Monday July 29, 2024 @12:02PM (#41101449)
Great comment! My wrong comment at least served as a MacDonal's theory https://jonbell.medium.com/mcdonalds-theory-9216e1c9da7d

As an outsider CUDA is so intimidating so the promise of Triton etc is very appearing and I wanted to get sold.