It's the exact reason why I've avoided CUDA for years, but I hit a dead end with OpenGL and Vulkan, and CUDA happened to be a fantastic, easy and fast solution. Of course I don't want graphics programming to be NVIDIA-only, but I want it to be like CUDA, just for all platforms.
Something which requires you to buy new hardware from a specific brand, and load an out-of-tree binary-only module on your kernel? That's not what I want graphics programming to be like.
The Vulkan API might be clunkier (I don't know, I haven't looked at the CUDA API, since I don't have the required hardware), but at least it can work everywhere.