News for hackers. Stuff that matters.
Re:Tensorflow 2.0 AMD Support by danielscrubs on Thursday November 28, 2019 @09:48AM (#21657694)
float16 or bfloat16?
Re:Tensorflow 2.0 AMD Support by bonoboTP on Thursday November 28, 2019 @11:46AM (#21658614)
I wasn't aware of the difference, but now I looked. I've always used float16, not bfloat16 (tf.float16 is the usual float16), and it causes a large speed boost on Turing NVIDIA cards with tensor cores as opposed to float32.

But certainly bfloat16 sounds attractive with its larger dynamic range, as float16 requires workarounds like loss scaling to avoid rounding down to zero when the exponent gets too small.

Re:Tensorflow 2.0 AMD Support by bonoboTP on Friday November 29, 2019 @03:51PM (#21666737)
Seems like bfloat16 is only available for TPUs in TensorFlow[0-1].

[0] https://github.com/tensorflow/tensorflow/issues/21612

[1] https://github.com/tensorflow/tensorflow/issues/21317