Re:Volta: Advanced Data Center GPU
by paulsutter on Wednesday May 10, 2017 @08:12PM
(#14312903)
Yes but to the extent you can, it's an easy win. I switched to a GEMMable method for a preprocessing step today based on the Volta and recent TPU news.
Hopefully Tensorflow XLA or other optimization frameworks could solve this problem in a more general way in the medium term:
http://www.kdnuggets.com/2017/04/deep-learning-virtual-machi...
In supercomputing this is the problem with using high performance linpack for benchmarks, which typically exceeds actual scientific codes by an order of magnitude in terms of floating point operations per second.