Mainly they decided to manufacture their tech here in Mexico instead of in Asia. But here on Mex we dont have the skills to to such high tech manufacturing.
For the sake of argument, let’s reduce Nvidia (my employer but I don’t speak for them) to just a silicon vendor.
You’re not only ignoring the work of thousands who design the stuff that goes on that silicon, the billions of $ in R&D, you’re also dismissing the decisions about what to put in that design, way before other people do it. Some of those decisions were made years before the actual implementation started. It happens to be something a few people at Nvidia are very good at.
There were times in the past where others had a higher number of FLOPS, but it didn’t make a dent in the usage of CUDA.
My standard assumption about anything in tech (and beyond) is that you don’t win on first principles alone (“parallel design is well understood”). It’s the details that make you stand out.
Bill Daly gave a great presentation at Hot Chips a few weeks ago that explains how Nvidia’s AI performance was only in small parts due to process improvement. The rest was doing things smarter: matrix sparsity, different number representations and so forth.
They then had the good fortune that their competitors tried to leverage open source and cross company collaboration. While misc open source winning is sometimes thing, the industry built all their stuff on cuda instead of opencl or openmp.
Nvidia thus built their gold mine.
I don't think it's impenetrable. There are some design mistakes baked into their system that are going to be difficult to unravel without breaking backwards compatibility.
They don't make their own chips, right?
To your second point: I am surprised that AMD hasn't tried harder to chase the high-end enterprise GPU market. Maybe that is a longer term secret plan after they gain more CPU market share. It is possible that a GPU ecosystem cannot out-compete NVidia's CUDA at this point.