Re:Tensorflow 2.0.0-Alpha0
by dragontamer on Wednesday March 06, 2019 @03:46PM
(#19322585)
AMD's software stack is definitely holding them back. But the hardware in smaller tests I've done really can get close to the theoretical TFLOPs / Memory Bandwidth numbers that they tout.
NVidia made the decision to hop on the Clang bandwagon years ago, and clearly that's allowed them to more easily optimize their code and move things forward. AMD finally has hopped on Clang with ROCm (but unfortunately still has some kinks to work out, esp with OpenCL)... and the compiler output seems to be superior as a result.
-------------
It seems unlikely to get a value-proposition out of AMD if you're going for deep learning. NVidias libraries are simply superior at the moment, and they get superior performance as a result.
But when you look at lower-level software that is written for both GPUs, AMD's compute girth really shows. Cryptocoins are perhaps the best example, although not every work-load looks like mining.
Luxray and Blender are more pragmatic: software raytracers which demonstrate AMD's incredibly GPU girth and memory performance with HBM2. The AMD Vega chips can outperform NVidia's equivalent in these cases.
-----------
All in all, AMD's ROCm environment seems like it is ready for programmers who are willing to stick to the C (or HCC) level, coding their own libraries and maybe using a bit of inline assembly here or there to maximize performance.
CUDA has more libraries (BLAS: linear algebra, better Tensorflow support for Deep Learning, etc. etc.) and that's definitely a major selling point. Especially because of how well those libraries are written.
I guess what I'm trying to say is... AMD's hardware is fine. Its their software thats clearly holding them back.
NVidia made the decision to hop on the Clang bandwagon years ago, and clearly that's allowed them to more easily optimize their code and move things forward. AMD finally has hopped on Clang with ROCm (but unfortunately still has some kinks to work out, esp with OpenCL)... and the compiler output seems to be superior as a result.
-------------
It seems unlikely to get a value-proposition out of AMD if you're going for deep learning. NVidias libraries are simply superior at the moment, and they get superior performance as a result.
But when you look at lower-level software that is written for both GPUs, AMD's compute girth really shows. Cryptocoins are perhaps the best example, although not every work-load looks like mining.
Luxray and Blender are more pragmatic: software raytracers which demonstrate AMD's incredibly GPU girth and memory performance with HBM2. The AMD Vega chips can outperform NVidia's equivalent in these cases.
-----------
All in all, AMD's ROCm environment seems like it is ready for programmers who are willing to stick to the C (or HCC) level, coding their own libraries and maybe using a bit of inline assembly here or there to maximize performance.
CUDA has more libraries (BLAS: linear algebra, better Tensorflow support for Deep Learning, etc. etc.) and that's definitely a major selling point. Especially because of how well those libraries are written.
I guess what I'm trying to say is... AMD's hardware is fine. Its their software thats clearly holding them back.