Re:What intelligent machines need to incorporate from the ne...
by Seanny123 on Saturday June 03, 2017 @02:01AM
(#14475222)
Nice references! I have some criticism of "Toward an Integration of Deep Learning and Neuroscience" which I've talked to the authors about. I felt like they downplayed the importance of integration between neuroscience and machine learning [1], while over-playing the biological plausibility of Deep Learning [2]. I'm also uncomfortable with the characterization of Neuromorphic computing as "Deep Learning on a Chip", as this dismisses the possibility of online learning and neural dynamic systems [3], which I think are necessary for truly intelligent systems.
As for Liquid State Machines, those evolved into Reservoir Computing and Echo State Machines. If you want to read more about them, I would recommend this paper comparing them to the Neural Engineering Framework [4] (with code!) to get a good idea of the state of the field.
[1] https://medium.com/@seanaubin/deep-learning-is-almost-the-br...
[2] https://cogsci.stackexchange.com/q/16269/4397
[3] https://medium.com/@seanaubin/a-way-around-the-coming-perfor...
[4] https://github.com/arvoelke/delay2017/blob/master/delay2017....
Re:What intelligent machines need to incorporate from the ne...
by return0 on Saturday June 03, 2017 @04:03AM
(#14475478)
Definitely people are working, but we dont have an addition to our methods of something fundamental. The papers you re referencing provide interesting conceptual theories, but none of them will survive for decades the way Hodkin-huxley's or Rall's models did. A lot of work on dendrites goes back to the 90s (e.g. seminal work of Mel http://www.nature.com/neuro/journal/v7/n6/abs/nn1253.html) and we still dont have a convincing theory of what they do.
I wouldn't really say that. There's a lot of progress being made in using and understanding biological processes for useful tasks. Not surprisingly, the biological mechanisms are surprisingly complex and rich and vary a lot through the brain (like you mentioned with the dendrites. Dendrites also work like coincidence detection mechanisms sometimes in some layers of the cortex [1] for instance)
[2] gives a very nice overview of what is needed from both machine learning and computational neuroscience to solve the entire problem of understanding the human brain.
There was the liquid state computing paper [3] in 2002 which showed that random networks of spiking neurons can perform some computations and have memory even in the absence of special learning rules.
There has also been quite some work on understanding e.g. the role of assemblies of neurons (groups of neurons firing), assembly sequences, plasticity rules, theory formulating neural activity in a probabilistic manner, a better understanding of the role of inhibition, dendrites etc.
And lastly, there have been huge advances in neuromorphic computing, both digital and analog. In some cases, the performance on these chips (that use spiking neurons) approaches that of state of the art machine learning. e.g. [4]
[1] http://www.sciencedirect.com/science/article/pii/S0166223612...
[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5021692/
[3] http://www.mitpressjournals.org/doi/abs/10.1162/089976602760...
[4] http://www.pnas.org/content/113/41/11441.abstract