News for hackers. Stuff that matters.
Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by mooneater on Wednesday April 05, 2017 @01:34PM (#14043868)
"This first generation of TPUs targeted inference" from [1]

So they are telling us about inference hardware. Im much more curious about training hardware.

[1] https://cloudplatform.googleblog.com/2017/04/quantifying-the...

Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by cec on Thursday April 06, 2017 @05:06AM (#14049002)
The paper says "we started a high-priority project to quickly produce a custom ASIC for inference (and bought off-the-shelf GPUs for training)."
Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by sgk284 on Wednesday April 05, 2017 @02:53PM (#14044555)
Using approaches like OpenAI's recent evolution strategies paper would remove the need for backprop, likely allowing these TPUs to be using for training without any changes.
Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by p1esk on Wednesday April 05, 2017 @03:12PM (#14044705)
Evolution strategy method is used in reinforcement learning models. How are you planning to use it for supervised learning?
Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by mdda on Thursday April 06, 2017 @01:40AM (#14048216)
People have known that training NNs (for any purpose) using evolution works well since the 1990s. The rise of the NN frameworks has made doing differentiation much easier now than it was before (and having gradient hints is intuitively a good idea). But for OpenAI to allow their PR people to declare this as a novel advance is ... surprising.
Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by p1esk on Thursday April 06, 2017 @12:15PM (#14051909)
Citation for training NN for image classification task where evolution works well?

Let's say you want to use a genetic algorithm to find a good set of weights: you generate, mutate, combine and select many random networks, and repeat this process many times. How many networks and how many times? That depends on the length of your chromosome and complexity of the task. Networks that work well for image classification need at least a million weights. The entire set of weights is a single chromosome. You realize now how computationally intractable this task is on modern hardware?

Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by mdda on Friday April 07, 2017 @01:05AM (#14057260)
> NN for image classification task

You've created your own straw man here.

> "You realize now how computationally intractable this task is on modern hardware?"

Here are the people that prove it isn't computationally intractable : https://blog.openai.com/evolution-strategies/ - but to say they've discovered a new breakthrough method is over-selling the result.

Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by p1esk on Friday April 07, 2017 @01:15AM (#14057321)
You said: "training NNs (for any purpose) using evolution works well". I gave you an example of a purpose where it does not work well. So, let me ask you again: can you give an example of evolutionary methods that work well when applied to training NNs, other than this breakthrough by OpenAI, which only works for RL?
Re:An In-Depth Look at Google's Tensor Processing Unit Archi... by monk_e_boy on Wednesday April 05, 2017 @02:51PM (#14044538)
GPUs I would guess. Great floating point speed.