How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Performance: A Worklog

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by ladberg on Friday July 26, 2024 @04:33PM (#41082029)

If this were true (and I highly doubt it) it's obvious how to make money from it: collect a 7 figure paycheck from Nvidia, AMD, or any FAANG.

[ Parent | on HN ]

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by ap4 on Friday July 26, 2024 @04:44PM (#41082129)

I swapped one of the kernels in the code from the article to my kernel, and left only the multiplication of matrices of size 4096².

On average over 20 runs:

CuBLAS (./sgemm 0) has 50.9 TFLOPS.

My kernel has 61.8 TFLOPS, so it's actually +21% speedup in this benchmark.

How do I collect my paycheck?

[ Parent | on HN ]

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by aaa370 on Friday July 26, 2024 @05:17PM (#41082354)

I gotta see it to believe it ;)

[ Parent | on HN ]

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by ap4 on Saturday July 27, 2024 @03:45PM (#41088847)

Believe it or not.

On a 4090 gpu, average of 20 runs of SGEMM_CUDA:

  size    tflops_cublas  tflops_my  diff
  4096²   50.8-50.9      61.8       +21%
  8192²   56.3-56.4      67.1       +19%
  16384²  53.6           66.7       +24%

I guess the right thing to do now would be to hire a B2B salesman and figure out, which company needs it.

[ Parent | on HN ]

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by ap4 on Wednesday July 31, 2024 @09:10PM (#41125271)

For all doubters, I open-sourced it: https://github.com/arekpaterek/Faster_SGEMM_CUDA

[ Parent | on HN ]

Re:How to Optimize a CUDA Matmul Kernel for CuBLAS-Like Perf... by JuanJohnJames on Friday July 26, 2024 @05:01PM (#41082238)

Post the code and your curriculum

[ Parent | on HN ]