News for hackers. Stuff that matters.
Re:Intel Discontinues the Intel Developer Forum; IDF17 Cance... by paulsutter on Monday April 17, 2017 @03:20PM (#14133653)
There's nothing about the TPU architecture that prevents use for training. Training is also done as tensor operations, and training can be done at limited precision[1]. Just because the first TPU was memory-limited doesn't mean TPUs can never be used for training.

[1] https://arxiv.org/abs/1606.06160

Re:Intel Discontinues the Intel Developer Forum; IDF17 Cance... by pjscott on Monday April 17, 2017 @07:15PM (#14135249)
The key problem with the TPU architecture for training is that its on-die buffer space is fairly limited and its memory bandwidth is currently nowhere near that of a GPU. This matters if you've got a lot of intermediate buffers that need to be alive at the same time for backpropagation. Also, there's the less fundamental matter of convenience: while there are some clever ways to train with limited precision gradients, it's not yet as easy as not doing that.

One style of training that would be a good fit for the current TPU architecture is some of the recent work on reinforcement learning without gradients:

https://arxiv.org/abs/1703.03864

Re:Intel Discontinues the Intel Developer Forum; IDF17 Cance... by shaklee3 on Monday April 17, 2017 @04:38PM (#14134211)
Sure, but it's a garbage in, garbage out thing. If you're using those in production in something like, say, google images, you wouldn't want to lose accuracy on a one-time pass. Anything can be quantized lower, but you start making the tradeoff of losing information that may affect the quality of your product later.