Re:Intel Discontinues the Intel Developer Forum; IDF17 Cance...
by pjscott on Monday April 17, 2017 @07:15PM
(#14135249)
The key problem with the TPU architecture for training is that its on-die buffer space is fairly limited and its memory bandwidth is currently nowhere near that of a GPU. This matters if you've got a lot of intermediate buffers that need to be alive at the same time for backpropagation. Also, there's the less fundamental matter of convenience: while there are some clever ways to train with limited precision gradients, it's not yet as easy as not doing that.
One style of training that would be a good fit for the current TPU architecture is some of the recent work on reinforcement learning without gradients:
One style of training that would be a good fit for the current TPU architecture is some of the recent work on reinforcement learning without gradients:
https://arxiv.org/abs/1703.03864