Re:New chip reduces neural networks’ power consumption
by arcticbull on Thursday February 15, 2018 @03:01PM
(#16387064)
DACs on one end and ADCs on the other, yeah. I guess though, it may be possible to hook analog inputs directly to it in some way, such as a camera or mic.
Re:New chip reduces neural networks’ power consumption
by deepnotderp on Thursday February 15, 2018 @08:49PM
(#16389310)
1. Yes, and the advantage of analog is significantly reduced as well, as a binary multiply and add then reduce to single gate operations.
2. Switched capacitor multipliers (actually almost anything analog) have difficulty scaling with feature shrinks, and that's before considering noise. E.g. see all the difficulties Xilinx went through to get 16nm FinFET ADCs efficiently working.
Re:New chip reduces neural networks’ power consumption
by phkahler on Thursday February 15, 2018 @08:08PM
(#16389172)
Right, they need to compare it to the same algorithm running on a conventional computer (or GPU or whatever they compared to). Math like that can run faster and more efficiently on a regular CPU too.
Re:New chip reduces neural networks’ power consumption
by synthos on Thursday February 15, 2018 @01:56PM
(#16386512)
It's not really comparable to TPU. The MIT chip uses analog tricks to perform binary neural network operations. It's probably directly comparable to FPGA DNN, I think.
Re:New chip reduces neural networks’ power consumption
by jaredhansen on Thursday February 15, 2018 @01:55PM
(#16386510)
Do read the whole thing, but here's the quick summary of most important parts:
[neural net] “nodes” are just weights stored in a computer’s memory. Calculating a dot product usually involves fetching a weight from memory, fetching the associated data item, multiplying the two, storing the result somewhere, and then repeating the operation for every input to a node...
In the [new] chip, a node’s input values are converted into electrical voltages and then multiplied by the appropriate weights. Summing the products is simply a matter of combining the voltages. Only the combined voltages are converted back into a digital representation and stored for further processing.
The chip can thus calculate dot products for multiple nodes — 16 at a time, in the prototype — in a single step, instead of shuttling between a processor and memory for every computation...
One of the keys to the system is that all the weights are either 1 or -1... Recent theoretical work suggests that neural nets trained with only two weights should lose little accuracy — somewhere between 1 and 2 percent.
Biswas and Chandrakasan’s research bears that prediction out. ... Their chip’s results were generally within 2 to 3 percent of the conventional network’s.
Re:New chip reduces neural networks’ power consumption
by newen on Thursday February 15, 2018 @02:21PM
(#16386696)
If the weights are only 1 or -1, and each neuron can only get 16 inputs, then this is more similar to neuromorphic systems like IBM's True North than regular neural networks.
And True North consumes on the order of 70-100 milliwatts.
2. CDAC + switched capacitor multiplier + SAR ADC can be competitive with digital multiplier.
2. Switched capacitor multipliers (actually almost anything analog) have difficulty scaling with feature shrinks, and that's before considering noise. E.g. see all the difficulties Xilinx went through to get 16nm FinFET ADCs efficiently working.
I'd be interested on a more conservative change using a single byte or maybe 4 bits.
[neural net] “nodes” are just weights stored in a computer’s memory. Calculating a dot product usually involves fetching a weight from memory, fetching the associated data item, multiplying the two, storing the result somewhere, and then repeating the operation for every input to a node...
In the [new] chip, a node’s input values are converted into electrical voltages and then multiplied by the appropriate weights. Summing the products is simply a matter of combining the voltages. Only the combined voltages are converted back into a digital representation and stored for further processing.
The chip can thus calculate dot products for multiple nodes — 16 at a time, in the prototype — in a single step, instead of shuttling between a processor and memory for every computation...
One of the keys to the system is that all the weights are either 1 or -1... Recent theoretical work suggests that neural nets trained with only two weights should lose little accuracy — somewhere between 1 and 2 percent.
Biswas and Chandrakasan’s research bears that prediction out. ... Their chip’s results were generally within 2 to 3 percent of the conventional network’s.
And True North consumes on the order of 70-100 milliwatts.