This was, if anything, the main theme at Supercomputing: “there is not even a point in talking about Nvidia’s new benchmarks. You all are going to buy it anyways” coupled with very few “we are betting on buying cheap hardware (Intel/AMD GPUs) and hope that we can build the relevant parts of CUDA ourselves” and the latter is pretty much a desperation move of labs/sites that simply cannot get NVIDIA GPUs (either price or availability).
And yes that is probably the seed of the end of Nvidia’s dominance. But it will take 20 years and multiple fuck ups. Just as it did with Intel
This could be 100% true, and the best ML engineers in the world could be completely oblivious to CUDA and it wouldn't change why nvidia is leading right now.
Anyone who thinks it's "trivial" to use AMD cards should help geohot and his tinygrad boxes. Despite emailing the CEO several times, it was a clear uphill battle to get something working, and even now the sales page lists driver quality as "mediocre".
If you open a GitHub issue or a forum post about your weird CUDA application that isn't working right, you will have an engineer help you. This is in nvidia's DNA and they did this before AI started printing money. On the flipside, you can email Lisa Su today and still not have problems fixed in AMDs drivers. I'm sure AMD is trying, this isn't to say they aren't, but having decent software culture at these large companies is hard.
If AMD can't do it, I feel even less confident about Intel. If the CUDA moat truly didn't matter, we would see AMD cards in demand. However it doesn't and it isn't for lack of trying.
That doesn’t mean, that ML people don’t care about some hardware specifics btw. Of course they do! Because CUDA is very, very good at exploiting the specific features of the newer cards (I’ve forgotten, what exactly this was though). But take away the simple integration into their favorite AI-SDK and it would be completely irrelevant.
The point is just that, if Intel were to build strictly better hardware, but not provide the interface to torch, then they would still loose to Nvidia. Yes, yes, this is not completely true. Given enough advantage someone else would build the IntelCUDA. But this is literally the bet that those academics make. And it’s so risky, that only academics are doing that
AI hasn't cared about Cuda for years - sure, Torch on NV will use it, but AI is all about Torch/TF, not details below that.