The GP part stands for "something else than graphics". CUDA is not general purpose in the sense that a lot of types of computation and data movement results in terrible performance. You buy GPU accelerators for performance reasons.

Case in point, GPUs are standard (for now) for training DNNs, but the big server farms that actually do the scoring run on Xeons.

What happened to OpenCL?

https://en.wikipedia.org/wiki/OpenCL

Nothing per se. It's just that it lags far enough behind CUDA that unless you have a specific reason to avoid NVIDIA hardware there are very few pragmatic reasons to use it.

Google has implemented their own deep learning chip with no CUDA support and its serving their purposes well. Why not another industry giant, Intel?

Google is very different from Intel on these fronts at least: 1. Google's open framework TensorFlow is exploding in popularity effectively moving the common interface up from CUDA. 2. Google has a huge need for internal AI/DeepLearning 3. Google has cloud and AI/DeepLearning as a service business 4. Google doesn't sells hardware

However, Intel dominance is not to be underestimated, they definitely can make industry wide impact quickly. Just saying you can not easily draw parallels here

What's stopping them provinding a backend for tensorflow? But yes'm, it's obviously not a given.