You can even see it in author's comments in the original article:

"When I first looked at fp16 Inception3 was the largest model I could train. Inception4 blew up until I went back to fp32. Mixed precision needs extra care, scaling of gradients and such. Still I think it is a good thing. What I really want to test is model size reduction for inference with TensorRT targeted to tensorcores. I think that is probably the best use case. Non-linear optimization is just too susceptible to precision loss."

There was also some NVidia video presentation recommending mixed FP32/FP16 training instead of pure FP16.

Mixed precision training can give you tensor core speedups. Paper: https://arxiv.org/abs/1710.03740 Toolkit which implements it on top of Tensorflow: https://github.com/NVIDIA/OpenSeq2Seq