Many state-of-art models won't train well on FP16. But for inferencing it's extraordinarily good. 2x1080Ti is the sweet spot for FP32 training on "budget" at the moment.

Got any sources? Was thinking about buying one just for the tensor cores, but if this is the case I probably won't.

You can even see it in author's comments in the original article:

"When I first looked at fp16 Inception3 was the largest model I could train. Inception4 blew up until I went back to fp32. Mixed precision needs extra care, scaling of gradients and such. Still I think it is a good thing. What I really want to test is model size reduction for inference with TensorRT targeted to tensorcores. I think that is probably the best use case. Non-linear optimization is just too susceptible to precision loss."

There was also some NVidia video presentation recommending mixed FP32/FP16 training instead of pure FP16.

Mixed precision training can give you tensor core speedups. Paper: https://arxiv.org/abs/1710.03740 Toolkit which implements it on top of Tensorflow: https://github.com/NVIDIA/OpenSeq2Seq

"Inference" or "inferring". "Inferencing" isn't a word any more than "defencing" is.

"Inference" is a term of art in machine learning jargon so those replacements you propose have much broader meanings than the original and are not suitable replacements.

This is outrageous. Consider math terminology as a so-called "term of art." An attractor, in dynamic systems, is something which attracts. A repellor is something which repels. Should it be called a "repellence?" Wouldn't that mean something undesirable? Should the opposite of "attractor" be a "repugnor," because repugnance is the opposite of attraction? What is the corresponding correct form of something that repels? Repellor (or repeller, for those who prefer American suffixes).

Forms matter. Colloquial meanings also matter, but not as much, particularly when they're an egregious violation of English and decency.