Re:Benchmarking State-Of-the-Art Deep Learning Software Tools
by T-A on Saturday August 27, 2016 @09:27AM
(#12372250)
Context-switching is expensive. You have to swap out the data being worked on by thread #1 and swap in the data for thread #2. So you end up being bottlenecked by memory bandwidth and latency rather than by raw compute.
You're right that Torch is faster than TensorFlow in RNN. But Torch is slower than TesnorFlow in AlexNet and ResNet. There is a set of benchmarks for many DL approaches as found in https://github.com/soumith/convnet-benchmarks