News for hackers. Stuff that matters.
Re:A new GPU backend for the TVM stack by dragontamer on Monday October 30, 2017 @12:58PM (#15586682)
Hmm, with the "Tensor Cores" of NVidia's next-generation Volta coming in, I'd bet that NVidia cards will be faster in machine learning tasks.

But then again: AMD Vega 56 / 64 have HBM2 and are under $1000. IIRC, the Vega Frontier Edition is $999 and 16GB of HBM2 at 480GB/s theoretical bandwidth.

NVidia also has an offering with high-speed HBM2 RAM: The Tesla P100, but its way more expensive: $7000 each.

I dunno if there are major benefits of HBM2 over GDDR5x however. Just listing off numbers here. The Titan Xp apparently has more bandwidth from the GDDR5x RAM for example, although the Titan XP is still more expensive than the Vega 64.

------------

If there is some problem that is global memory-bandwidth constrained, then it might be better to run it on AMD Vega 64. After all, you can pretty much afford 7x AMD Vega Frontier editions than the NVidia P100.

Obviously, this very much depends on your workload.

Re:A new GPU backend for the TVM stack by dharma1 on Monday October 30, 2017 @02:47PM (#15587784)
Yep, Nvidia is quoting 125 TFLOPs mixed precision on V100, boosted by Tensor Cores.

Vega 64 can in theory do 25 TFLOPs half precision.

But as you say there's a large price difference too.

For a market segment that needs 1-8 GPU rigs for ML on a low budget AMD could kill it if they invested in software support and kernel optimisation.

For servers and large scale training, unless AMD has some ML specialised cores in the pipeline, Nvidia Volta and Google TPUs have a serious lead.

Re:A new GPU backend for the TVM stack by mtgx on Monday October 30, 2017 @03:34PM (#15588164)
Aren't the Tensor Cores mostly for inference, not training?
Re:A new GPU backend for the TVM stack by dragontamer on Monday October 30, 2017 @04:17PM (#15588500)
The Volta Tensor Cores aren't released yet, and I haven't played with Google's TPUs at all.

But NVidia markets Tensor cores as:

> New Tensor Cores designed specifically for deep learning deliver up to 12x higher peak TFLOP/ss for training, and 6x higher peak TFLOP/s for inference

https://devblogs.nvidia.com/parallelforall/cuda-9-features-r...

I wouldn't be surprised if they were inflating the numbers slightly, as is common in a lot of marketing material.

Re:A new GPU backend for the TVM stack by dharma1 on Monday October 30, 2017 @04:33PM (#15588624)
You can use Tensor Cores on AWS p3 instances (v100) now. Just doesn't seem to be a whole lot of ML framework support for mixed precision training with CUDA 9 yet

https://devblogs.nvidia.com/parallelforall/programming-tenso...

Google's TPUs also do low precision training (with some special version of TF). https://cloud.google.com/tpu/