Re:New Compute Engine A2 VMs–First Nvidia Ampere A100 GPUs i...
by g_airborne on Tuesday July 07, 2020 @06:06PM
(#23764655)
Very cool! Does anyone know how is software support for all these features? It seems that TF doesn’t support the TP32/16 types as of yet. Is this something only CUDA engineers can use right now?
It does seem a little fishy to me that NVIDIA often boasts with figures like 10x performance upgrade whilst in practice those are only possible if you use one of their non-default float types which are hardly supported in most deep learning libraries :(
Re:New Compute Engine A2 VMs–First Nvidia Ampere A100 GPUs i...
by zetazzed on Tuesday July 07, 2020 @09:49PM
(#23766178)
Both PyTorch and Tensorflow teams have announced they'll support TF32. By design, it interoperates well with existing code, since calling code can just treat it as a regular FP32 value.
Lots of architectural changes like MIG, new floating point formats, etc. Great to see GCP getting VMs out pretty soon after launch so people can start kicking the tires.