Re:A new GPU backend for the TVM stack
by mtgx on Monday October 30, 2017 @03:34PM
(#15588164)
Aren't the Tensor Cores mostly for inference, not training?
[ Parent
| on HN ]
Re:A new GPU backend for the TVM stack
by dragontamer on Monday October 30, 2017 @04:17PM
(#15588500)
The Volta Tensor Cores aren't released yet, and I haven't played with Google's TPUs at all.
But NVidia markets Tensor cores as:
> New Tensor Cores designed specifically for deep learning deliver up to 12x higher peak TFLOP/ss for training, and 6x higher peak TFLOP/s for inference
https://devblogs.nvidia.com/parallelforall/cuda-9-features-r...
I wouldn't be surprised if they were inflating the numbers slightly, as is common in a lot of marketing material.
You can use Tensor Cores on AWS p3 instances (v100) now. Just doesn't seem to be a whole lot of ML framework support for mixed precision training with CUDA 9 yet
https://devblogs.nvidia.com/parallelforall/programming-tenso...
Google's TPUs also do low precision training (with some special version of TF). https://cloud.google.com/tpu/