That is a part of the problem: they assigned people for the task, and produced open-source libraries for matrices, FFT, maybe even something for DNNs. But, those are not polished much, and you have to hunt them down and install them yourself. And, they do a really bad job at marketing.

On the other hand, finding and installing those libraries is nothing compared to actually developing GPU computing software, so, as I said, if you want to program GPUs, even such scattered state of AMD platform is not that worse than Nvidia. Because, in Nvidia, you install CUDA and you get everything set up. And then - what do you do? You still have to learn a not-so-easy black art of optimizing kernels for the actual hardware.

I haven't seen CuDNN equivalents (in terms of perf) for common machine learning frameworks from AMD. I don't think they exist, if they did, people would shift to using AMD.

For NVidia I have seen some faster kernels than the ones supplied by NVidia - https://github.com/NervanaSystems/neon - though CuDNN introduced Winograd kernels too in their last update

I do not know about the quality of this since I do not use NNs, but there is https://github.com/hughperkins/DeepCL and I think I have seen others.