News for hackers. Stuff that matters.

oldmanLecun

Re:Yann LeCun on the IBM neural net chip on Saturday August 09, 2014 @05:10PM (#8158021)
From my understanding, the chip is geared towards implementing neural networks with low power consumption, which makes using spikes a reasonable design choice. So one can argue that using spikes is not about biological accuracy, but power efficiency.

That is great that people want general purpose machine learning chips, the question is how to do it in low power. My guess is that the right architecture will be a mix of ML primitives mixed in with things like spikes (and perhaps other primitives seen found in biology).

Re:Yann LeCun on the IBM neural net chip on Saturday August 09, 2014 @04:27PM (#8157925)
What LeCun is missing is that the IBM chip was designed to be ultra low power (63mW to do pattern recognition on a real time video). One of the novel techniques this chip uses to achieve low power is to use spikes to send information between neurons.

It is not clear how LeCun would propose to send information in his convolutional neural networks in specialized hardware (from what I can tell, he uses FPGAs which are very high power in comparison). In the worst case, that approach sends data between neurons on each time step, which is very inefficient in power. If they do something more clever, I would bet it would start to like sending spikes.

It also seems very short sighted to say that if the hardware is not specifically designed for convolutional neural networks, then it is not the right architecture. It seems like the IBM chip does support convolutional networks, but it might require a few extra time steps to average.

Biology has evolved to use spikes (across many different species). Perhaps evolution didn't get the memo that non-spiking convolutional neural networks is the only architecture worth building. Maybe it will take a few more thousand generations before evolution catches up, but until then spiking neuron architectures seem like a decent gambit ...