The new GPU of interest for DeepLearning is the MI60. It has 32GB of RAM, 1TB/s memory bandwidth. Performance on fp32 looks slightly lower than the V100 (resnet50 on tensorflow, see below). But there are no tensorcores, so resnet50 would go 3x faster using the TensorCores. Personally, i think TensorCores are not that useful in the general case, as not enough programmers that i have come across are writing their code to use them. They are good, but nichey.
The important thing we don't know is the price. Expect it to be released by the end of Q1 or early Q2.
Reference: https://wccftech.com/amd-radeon-mi60-resnet-benchmarks-v100-...
I want AMD to win, I really do, but I can't personally justify taking chances on them anymore.
They're just fine, though I think a tuned implementation on either tends to show NVIDIA ahead.
> I have only heard of Nvidia CUDA in this context. Does anybody do ML work on AMD GPUs?
NVIDIA is still king here, but if AMD continues to develop their Radeon Instinct stuff, it could be competitive for some applications; it then becomes a question of whether or not there will be enough people who know something other than CUDA (which is not currently supported on AMD GPUs, though work is under way to make a compatible runtime).
That stack is also open source. And writing in it is supposed to keep you compatible with NVidea.
Wanting to support them -- and hoping they become more competitive in ML -- are why I bought Vega GPUs.
Although, I haven't found much time to actually try and get any of my software running on a GPU, let alone optimized. As a huge fan of avx512, I would like to try -- maybe I can get even better performance on a graphics card.
But with AMD, the lack of support for the uninitiated is apparent. Not much in the way of existing software and resources like online tutorials. I'd like guides on how to optimize kernels, organizing wavefronts and discussing memory movement. Maybe I haven't looked hard enough.
Especially painful in Julia, where almost half a year after 1.0, the only supported way to use GPUs on any library is with CUDANative. "Cross platform" GPUArrays.jl's open cl backend still hasn't been updated. Means all my coding will be done in HIP. Which is fine.
So yes you can use AMD hardware for it, but you end up making a lot more work for yourself to do so, and for little to no benefit to yourself.
As of today, if installing pytorch w/ GPU option the assumption is you have a nvidia gpu. Same thing seems to be the case for Tensorflow (did not install tensorflow lately)
The first mover effect of Nvidia/CUDA should not be under-estimated.
I have only heard of Nvidia CUDA in this context. Does anybody do ML work on AMD GPUs?
Does the popular libraries like PyTorch, TensorFlow etc support both CUDA and AMD's equivalent?