Re:2080 RTX performance on Tensorflow with CUDA 10
by bigmit37 on Friday October 05, 2018 @10:31AM
(#18148394)
Are there benefits to using FP32 vs FP16? I’ve been dabbling with deep learning but not really sure how much affect higher precision is having. Though more precision is better I suppose.
[ Parent
| on HN ]
Re:2080 RTX performance on Tensorflow with CUDA 10
by bitL on Friday October 05, 2018 @10:45AM
(#18148529)
Traditionally Deep Learning frameworks were all using FP32.
With FP16 one can theoretically get 2x speed and 2x larger models with the same VRAM capacity. For inferencing with INT8/INT4 it can be even way better (good for embedded stuff). The downside is that sometimes more complex/deep models don't converge (or converge less often than FP32). Sometimes there are framework issues with some advanced FP16 stuff.
Re:2080 RTX performance on Tensorflow with CUDA 10
by visionscaper on Friday October 05, 2018 @06:05PM
(#18152112)
From experience I know that models using RNNs have trouble training with FP16 precision. The common solution is to do training in FP32 and inference in FP16. To make this happen you often have to implement custom code (e.g. using Tensorflow or Keras as a meta framework)