Re:Intel Discontinues the Intel Developer Forum; IDF17 Cance...
by pjscott on Monday April 17, 2017 @07:15PM
(#14135249)
The key problem with the TPU architecture for training is that its on-die buffer space is fairly limited and its memory bandwidth is currently nowhere near that of a GPU. This matters if you've got a lot of intermediate buffers that need to be alive at the same time for backpropagation. Also, there's the less fundamental matter of convenience: while there are some clever ways to train with limited precision gradients, it's not yet as easy as not doing that.
One style of training that would be a good fit for the current TPU architecture is some of the recent work on reinforcement learning without gradients:
Re:Intel Discontinues the Intel Developer Forum; IDF17 Cance...
by shaklee3 on Monday April 17, 2017 @04:38PM
(#14134211)
Sure, but it's a garbage in, garbage out thing. If you're using those in production in something like, say, google images, you wouldn't want to lose accuracy on a one-time pass. Anything can be quantized lower, but you start making the tradeoff of losing information that may affect the quality of your product later.
[1] https://arxiv.org/abs/1606.06160