CUDA is the 'open' standard that exists today, has market/mind share, and doesn't lock you into a vendor. Look at google to see how difficult it is to make a new chip and surrounding system.
Still, Nvida should work on making the generation of chips and languages that come after CUDA.
It's nice to see companies like Apple adding CUDA support in their own way.
The thing that increasingly intrigues me is how long it might take AI-assisted development to explore what else might be possible in the space.
1. Physical infrastructure - power, cooling, network fabric, chip fabrication
2. Chips
3. Low level sw - CUDA/ROCm/Metal/XLA etc
4. Kernels - cuBLAS, cuDNN, FlashAttention, FlashInfer, Triton, llama.cpp
5. Kernel gen/optimization - MLIR, TVM, TensorRT
6. ML frameworks pytorch, jax, tensorslow
7. Models - Llama, Qwen, Claude, GPT, Gemini
8. More tuning - vLLM, SGLang, TensorRT-LLM, DeepSpeed, Megatron-LM
9. API/SDK/Platform
10. Apps/Agents/Pipelines/Orchestration etc
Aren't the cutting edge fabs entirely booked out?
Dylan Ratigan went on that rant a decade ago about it being a mathematical fact American wealth was being extracted through sale of all these corps and offshoring
We are to be serfs of royalty, religious cranks, and the rest of the non-contributor orator class
Maybe in the long term. The biggest threat in the short term is Nvidia's exposure to the data center bubble, which was already showing signs of popping before Jensen's departure. Given Nvidia's role as a key investment bank in the AI trade, making and brokering circular financing deals around data centers, his departure could hasten the data center collapse if the company starts backing away from its key financing role.
I do agree CUDA is becoming less relevant though.
There are two companies I'd be most concerned about if I were NVidia's CEO:
1. Apple. It's huge and has a long history at this point of making very efficient chips that use a shared memory pool. This defeats NVidia's tactic of aggressive market segmentation by the amount of VRAM. You can buy a Mac Studio with 256GB of RAM (well, you have to wait until March currently) for ~$11k. That's a lot cheaper than a single RTX 6000 Pro; and
2. Google. They currently buy or rent a lot of GPU capacity from SpaceX and others as well as make their own data centers. But they also have their own TPUs that they've been developing for years. I would be shocked if they've not investing billions into ramping up that program so they can use homegrown chips for AI. Google is still the best at building their own data centers and their own networking is industry-leading (IMHO).
Google is putting billions into TPUs and this isn’t a secret.
> so they can use homegrown chips for AI
While not likely assumed, it’s no secret that TPUs have been in use for training and inference for a very long time.
I'd rephrase that a bit. The biggest threat was from their customers commoditizing them (while Google and Amazon could become competitiors, rest are way more iffy). You know, nVidia actually competing on price instead of relying on locking them in nVidia propietary landscape.
https://www.wsj.com/tech/ai/nvidias-days-as-an-open-checkboo...
Surely Nvidia won’t stand still and will continue to compete, but the margins will collapse along with volumes.
AMD will probably suffer less, relatively speaking. Their failure to invest in software finally seems to be paying off now that LLMs can pick up their slack. But the growth currently priced in to their stock will never materialize and they’ll be seen as arriving just in time for the lights to go out.