Re:Jim Keller criticizes Nvidia's CUDA, x86
by qwertox on Friday February 23, 2024 @10:40AM
(#39481921)
But what kind of CUDA kernels have you written? Kernels for AI training and inferencing? Because I think that's what he's trying to say.
I doubt he's saying that in the context of CFD or other similar ones.
Re:Jim Keller criticizes Nvidia's CUDA, x86
by brucethemoose2 on Friday February 23, 2024 @10:43AM
(#39481963)
If you are implying we are at the mercy of a few CUDA experts because we can't efficiently write it ourselves, you would be correct.
Triton does seem way more approachable as something I'd like to pick up.
Re:Jim Keller criticizes Nvidia's CUDA, x86
by pjmlp on Friday February 23, 2024 @04:07PM
(#39486111)
Hence why NVidia is also investing into GPUs JITs for Python.
One advantage of CUDA, that Khronos/AMD/Intel realized too late, was the polyglot support that NVidia started to push and support since CUDA 3.0.
Re:Jim Keller criticizes Nvidia's CUDA, x86
by imtringued on Friday February 23, 2024 @06:00PM
(#39487252)
I hear you say this over and over again, but honestly AMDs OpenCL implementations used to be unusable garbage.
Re:Jim Keller criticizes Nvidia's CUDA, x86
by pjmlp on Saturday February 24, 2024 @01:29AM
(#39489603)
That in top of everything else, with OpenCL stuck in 1990's tooling.
Re:Jim Keller criticizes Nvidia's CUDA, x86
by PeterisP on Friday February 23, 2024 @03:43PM
(#39485849)
But why do I need to write CUDA kernels? The CUDA kernels I need already exist, and I don't particularly care if writing them was so horrible that it required scratching your eyes out and sacrificing kittens to demons - I would care about the (re)writing effort only if I was considering other hardware that doesn't yet have the support that CUDA has as then I'd actually have to do that myself.
Quite a thing to say as head of an AI hardware startup.
Most of the machine learning libraries we use/test in production (and that I use personally on my desktop) have hand written, extensively optimized CUDA kernels. It would be problem #1 switching to Tenstorrent hardware.
Not that I like that one bit, or that I really understand the optimization of these libraries. I do see triton code as well, but it still seems to be the suboptimal path.