Re:Apple M1 support for TensorFlow 2.5 pluggable device API
by 6gvONxR4sf7o on Tuesday June 08, 2021 @10:24PM
(#27443553)
I’ve been mostly using TPUs lately and non-TF32 GPUs at work, so I don’t have any practical experience with TF32, but the sales pitch seems pretty good. Do you have any personal experience on whether it’s as much of a drop in replacement for fp32 as they suggest?
[ Parent
| on HN ]
Re:Apple M1 support for TensorFlow 2.5 pluggable device API
by codelord on Tuesday June 08, 2021 @10:30PM
(#27443591)
I haven't used TF32 personally, but I think the sales pitch is not too far off. Most of the time I use mixed-precision training which should be similar to FP16/TF32 in terms of performance. It does tremendously speed up training.