Nvidia's moat for LLMs isn't Cuda, it's installed capacity, arrangements with frontier labs, TSMC bandwith, and just the fact that it currently has no real competition.
This Huang scandal has so many dimensions that are clearly more impactful on the world outside than to NVIDIA itself -- the blast radius seems to extend to the US cabinet -- that it begs questions if Jensen Huang wasn't id'd as the essential piece to remove to affect a comprehensive restructing in the segment.
So if deals fall apart because Jensen and his handshake have left the room, that may unclog many memory and processing supply lines that are currently spoken for by deals with Jensen. That may have been a goal from the start.
But once you do that, translating CUDA into one of many available vectorized backends is trivial.
We have a bunch of custom kernels for geometric operations (like merging polygons) and they were rather difficult to write a while ago. AI rewrote them in HIP during my lunch break.
Do we actually know they're more efficient per watt or for the power used and heat produced ? Or are they just easier to scale (and manufacturer) en masse ?
There's also the CUDA moat which is hard to get around