Re:Volta: Advanced Data Center GPU
by paulsutter on Wednesday May 10, 2017 @06:23PM
(#14312288)
Build your systems around GEMM/blas. Every vendor will give you a fast GEMM, and you'll be set for basically all the architectures that are coming out.
[ Parent
| on HN ]
Except that not all problems in computation are GEMMs. CNNs in Machine learning certainly are, but many 'real' systems cannot be posted in such a manner.
In supercomputing this is the problem with using high performance linpack for benchmarks, which typically exceeds actual scientific codes by an order of magnitude in terms of floating point operations per second.
Re:Volta: Advanced Data Center GPU
by paulsutter on Wednesday May 10, 2017 @08:12PM
(#14312903)
Yes but to the extent you can, it's an easy win. I switched to a GEMMable method for a preprocessing step today based on the Volta and recent TPU news.
Hopefully Tensorflow XLA or other optimization frameworks could solve this problem in a more general way in the medium term:
http://www.kdnuggets.com/2017/04/deep-learning-virtual-machi...