For researchers, yes. At the scale Facebook and Google operate at the lack of CUDA isn't much of a problem. The big difference is that AMD's GPUs only work on vectors whereas Google's TPUs (and NVidia's latest card) can work on matrices directly for a large increase in throughput.

Most DL researchers don't care about CUDA, they use higher level frameworks (Tensorflow, Pytorch, Caffe, etc). The frameworks can be extended to support other backends (e.g. OpenCL). If someone builds their own hardware, provides integrated support for that hardware in Tensorflow, and demonstrates speed advantages, people will use it.

This is something that I've wondered about. There is nothing about Tensorflow (for example) that is specific to CUDA. Why doesn't AMD dedicate a couple of engineers to port it? They don't even have to do a complete job. Only about half of the Tensorflow operations are used for most models.

AMD does. https://github.com/ROCmSoftwarePlatform/tensorflow

If they make CUDA the industry standard by porting it to AMD, and then Nvidia is still able to control the standard, over the long term AMD will always be in the position of having to catch up.

I think the GP is suggesting AMD port TensorFlow to OpenCL to break up the CUDA monopoly.

No need, there is already an industry standard that works across all hardware.

Why do you say that? Because they already have a large code base that would be too much work to move over? Or superior features in CUDA? This is actually related to some ongoing research for me, be very interested in any insight.

Because there's no compelling reason not to use CUDA.

So the problem is programmability