I don't personally think CUDA is impossible to replace - but I do think that everyone capable of replacing CUDA has been ignoring it recently. Nvidia's role as the GPGPU compute people is secure for the foreseeable future. Apple wants to design simpler GPUs, AMD wants to design cheaper GPUs, and Intel wants to pretend like they can compete with AMD. Every stakeholder with the capacity to turn this ship around is pretending like Nvidia doesn't exist and whistling until they go away.
So many billions of dollars and no one is even 1% close to displacing CUDA in any meaningful way. ZULDA is dead. ROCM is a meme, Scale is a meme. Either you use CUDA or you don't do meaningful AI work.
It's impossible to assail their monopoly without utilizing far lower prices, coming up under their extreme margin products. It's how it is almost always done competitively in tech (see: ARM, or Office (dramatically undercut Lotus with a cheaper inferior product), or Linux, or Huawei, or Chromebooks, or Internet Explorer, or just about anything).
Note: I never said lower prices is all you'd need. Who would think that? The implication is that I'm ignorant of the entire history of tech, it's a poor approach to discussion with another person on HN frankly.
In the case of ARM, Office, Linux, Huawei, and ChromeOS, these were all actual alternatives to the incumbent tools people were familiar with. You can directly compare Office and Lotus because they are fundamentally similar products - ARM had a real chance against x86 because wasn't a complex ISA to unseat. Nvidia is not analogous to these businesses because they occupy a league of their own as the provider of CUDA. It's not exaggeration to say that they have completely seceded from the market of GPUs and can sustain themselves on demand from crypto miners and AI pundits alone.
AMD, Intel and even Apple have bigger things to worry about than hitting an arbitrary price point, if they want Nvidia in their crosshairs. All of them have already solved the "sell consumer tech at attractive prices" problem but not the "make it complex, standardize it and scale it up" problem.
Intel seems to have thrown their weight behind SYCL, which is an open standard intended to compete with CUDA. Its not clear there has been much interest from other hardware vendors though.
You misunderstand why and how Nvidia is a monopoly. Many companies make GPUs, and all those GPUs can be used for computation if you develop compute shaders for them. This part is not the problem, you can already go buy cheaper hardware that outperforms Nvidia if price is your only concern.
Software is the issue. That's it - it's CUDA and nothing else. You cannot assail Nvidia's position, and moreover their hardware's value, without a really solid reason for datacenters to own them. Datacenters do not want to own GPUs because once the AI bubble pops they'll be bagholders for Intel and AMD's depreciated software. Nvidia hardware can at least crypto mine, or be leased out to industrial customers that have their own remote CUDA applications. The demand for generic GPU compute is basically nonexistent, the reason this market exists at all is because CUDA exists, and you cannot turn over Nvidia's foothold without accepting that fact.
The only way the entire industry can fuck over Nvidia is if they choose to invest in a complete CUDA replacement like OpenCL. That is the only way that Nvidia's value can be actually deposed without any path of recourse for their business, and it will never happen because every single one of Nvidia's competitors hate each other's guts and would rather watch each other die in gladiatorial combat than help each other fight the monster. And Jensen Huang probably revels in it, CUDA is a hedged bet against the industry ever working together for common good.