NVidia made the decision to hop on the Clang bandwagon years ago, and clearly that's allowed them to more easily optimize their code and move things forward. AMD finally has hopped on Clang with ROCm (but unfortunately still has some kinks to work out, esp with OpenCL)... and the compiler output seems to be superior as a result.
-------------
It seems unlikely to get a value-proposition out of AMD if you're going for deep learning. NVidias libraries are simply superior at the moment, and they get superior performance as a result.
But when you look at lower-level software that is written for both GPUs, AMD's compute girth really shows. Cryptocoins are perhaps the best example, although not every work-load looks like mining.
Luxray and Blender are more pragmatic: software raytracers which demonstrate AMD's incredibly GPU girth and memory performance with HBM2. The AMD Vega chips can outperform NVidia's equivalent in these cases.
-----------
All in all, AMD's ROCm environment seems like it is ready for programmers who are willing to stick to the C (or HCC) level, coding their own libraries and maybe using a bit of inline assembly here or there to maximize performance.
CUDA has more libraries (BLAS: linear algebra, better Tensorflow support for Deep Learning, etc. etc.) and that's definitely a major selling point. Especially because of how well those libraries are written.
I guess what I'm trying to say is... AMD's hardware is fine. Its their software thats clearly holding them back.
Here's the thing: In reality, it doesn't matter if a Radeon VII or Vega64 has better theoretical TFLOPs than a 1080 Ti or whatever if the card runs 50W hotter(!!!) and the software is bad enough it runs at 1/2 the performance of the competitor. 50W hotter at a 10% perf loss and a 40% decrease in price is attractive. Hotter at 1/2 the perf, is a non-starter (almost entirely ignoring price factors). Free drivers do not make up for this. DL is a fast enough field that, for most practitioners, your time is better served as an enthusiast/participant by just paying the extra upcharge on an NVidia card and getting on with your life, so you can actually train models and do your work.
There is only one viable solution if you want to "do your work" today -- not for better, and almost certainly for worst. Ceterum censeo.
DL moves fast enough that it's always possible for new frameworks to come around, so perhaps a competitor with a multi-vendor design, from the ground up, can break though. But then you have to ask whether a lot of these frameworks can even succeed/tread water in such an environment without Google/Facebook propping them up, which is a separate problem... And before that, AMD has to make sure HIP/HCC have to work reliably, every time, everywhere, and they have the libraries to back it up. I'm hoping the succeed at this.