Agree. But they also have a lot of hardware features. Mixed precision GEMM (fp8 and soon fp3), async copies and now TMA, now global shared memory. And with Blackwell they claim 50% less power, too. They are 10 years ahead of the competition and with a massive budget. I think they reached escape velocity.

Intel and Microsoft had the problem of competing with their previously sold products. Most of their users only needed a basic PC for email, Word, and Excel. Nvidia doesn't have this problem because previous hardware generations quickly become obsolete as the demand for compute keeps growing and it's easier to manage newer clusters.

s/fp3/fp4