CUDA not a moat? It’s physical hardware combined with extensive software libraries. How much more of a moat could they have?

In the end it's about who can do the most (probably low precision) FLOPS/$ - CUDA becomes less of a moat the more established the field becomes and the software settles. There's nothing magical about the low-level arithmetics that powers this.

Nvidia will have a first-mover advantage, but AMD will be strongly motivated to win those customers, and they should be able to, eventually.

Anyway, that's my gut instinct based on how these things tend to play out. Now let's hear from e.g. actual experts in the field? :)

There's a winner-take-all dynamic at play, though.

The better CUDA is, the more things get built on top of it.

The more things get built on top of CUDA, the better the CUDA ecosystem of tooling, frameworks, etc. gets.

The better the CUDA ecosystem is, the better CUDA is. GOTO 1.

But you're right -- lots and lots of folks will be motivated to compete.

Except that CUDA is low level, so it's not hard to shim above it and write interoperable code. There are too many players who don't want to play the Nvidia tax, this will play out like OpenGL vs Direct3d in reverse.

> this will play out like OpenGL vs Direct3d in reverse

Is that also like OpenCL vs CUDA in reverse?

I feel like abstractions don't work if we want to get the maximum performance. Afaik Tensor cores are not usable from opencl and on the other hand even in the CUDA universe cuBLAS (hand optimized) seems to outperform cutlass (using abstractions)

I dunno; CUDA is a library. How many closed source libraries win winner-takes-all games?

Nvidia is a lonely proprietary ship in a hostile sea of open source packages. It might take 20 years if Intel and AMD fail to do anything useful, but sooner or later they'll either open source CUDA defensively or get ground down by an open library. There are a finite list of features to implement before "Runs on Nvidia" vs "Runs on any graphics card" becomes the only important box left on the bureaucrat's checklist.

The math here is literally 1st & 2nd year university subjects. That is a good short term moat, but not a defensible one long term.

> I think AMD should be to able compete pretty well soon.

AMD has been trying to catch up in the machine learning space for many years now, and it hasn't happened. For things to finally improve, I guess something needs to change internally at AMD. Even Apple Silicon, in the short time it has existed, has gained better support for ML (e.g. compare running PyTorch on Apple Silicon vs running PyTorch on AMD).

100% this. Not seeing AMD make significant progress in ML against CUDA and other proprietary Nvidia libraries for years. The silicon is clearly capable but the software isn’t. Perhaps it’s time for leadership change in this division at AMD