News for hackers. Stuff that matters.
Re:Jim Keller criticizes Nvidia's CUDA, x86 by brucethemoose2 on Friday February 23, 2024 @10:27AM (#39481727)
> "Basically nobody writes CUDA,"

Quite a thing to say as head of an AI hardware startup.

Most of the machine learning libraries we use/test in production (and that I use personally on my desktop) have hand written, extensively optimized CUDA kernels. It would be problem #1 switching to Tenstorrent hardware.

Not that I like that one bit, or that I really understand the optimization of these libraries. I do see triton code as well, but it still seems to be the suboptimal path.

Re:Jim Keller criticizes Nvidia's CUDA, x86 by qwertox on Friday February 23, 2024 @10:40AM (#39481921)
But what kind of CUDA kernels have you written? Kernels for AI training and inferencing? Because I think that's what he's trying to say.

I doubt he's saying that in the context of CFD or other similar ones.

Re:Jim Keller criticizes Nvidia's CUDA, x86 by brucethemoose2 on Friday February 23, 2024 @10:43AM (#39481963)
If you are implying we are at the mercy of a few CUDA experts because we can't efficiently write it ourselves, you would be correct.

Triton does seem way more approachable as something I'd like to pick up.

Re:Jim Keller criticizes Nvidia's CUDA, x86 by pjmlp on Friday February 23, 2024 @04:07PM (#39486111)
Hence why NVidia is also investing into GPUs JITs for Python.

One advantage of CUDA, that Khronos/AMD/Intel realized too late, was the polyglot support that NVidia started to push and support since CUDA 3.0.

Re:Jim Keller criticizes Nvidia's CUDA, x86 by imtringued on Friday February 23, 2024 @06:00PM (#39487252)
I hear you say this over and over again, but honestly AMDs OpenCL implementations used to be unusable garbage.
Re:Jim Keller criticizes Nvidia's CUDA, x86 by pjmlp on Saturday February 24, 2024 @01:29AM (#39489603)
That in top of everything else, with OpenCL stuck in 1990's tooling.
Re:Jim Keller criticizes Nvidia's CUDA, x86 by PeterisP on Friday February 23, 2024 @03:43PM (#39485849)
But why do I need to write CUDA kernels? The CUDA kernels I need already exist, and I don't particularly care if writing them was so horrible that it required scratching your eyes out and sacrificing kittens to demons - I would care about the (re)writing effort only if I was considering other hardware that doesn't yet have the support that CUDA has as then I'd actually have to do that myself.