Why do CNNs generalize so poorly to small image transformations?

Re:Why do CNNs generalize so poorly to small image transform... by rdlecler1 on Sunday June 10, 2018 @02:55PM (#17279605)

This is probably the fifth or sixth trough. The problem is we spend too much time trying to brute force engineer our way forward when we should spend more time reverse engineering the salient properties that give rise to biological intelligence. Someone will invariably say that planes don’t use bird wings, but both birds and planes relying on the laws of aerodynamics and we need some kind of similar theoretical framework for computational intelligence—reverse engineering is the shortest path forward.

There are a few moves in this direction but without a better framework for computational intelligence, which we can get to faster through reverse engineering, we’re going to keep hitting these walls. This road includes methods for artificial neurogenesis, genetic encoding of the developmental program at the level of neuron and axon driven by artificial gene regulatory networks (AGRN) evolutionary methods to evolve populations, and adversarial ecosystems for open ended evolution. Unfortunately none of these will give you a quick win....

[ Parent | on HN ]

Re:Why do CNNs generalize so poorly to small image transform... by bitL on Sunday June 10, 2018 @04:09PM (#17279927)

> but both birds and planes relying on the laws of aerodynamics

Complexity of biological systems is so mind-blowing that we likely don't even have computational capacity to simulate one of those processes in veracious detail. In addition, we can't even comprehend primitive, 1000-layer deep neural networks, not mentioning significantly more complex neurons (that are performing some local, protein-based computations alongside electric spikes we measure). We should be happy that Deep Learning surprisingly somewhat works, obliterating "classical" ML and make use of it to the full extent of its capabilities. Romantic idea that we can construct AGI with it should be retired.

[ Parent | on HN ]

Re:Why do CNNs generalize so poorly to small image transform... by rdlecler1 on Sunday June 10, 2018 @10:26PM (#17281294)

This is lazy thinking. You could say the same thing about planetary motion but the ellipse equation does a pretty good job giving us a null hypothesis to build up from.

If you’re a materialist then you’re going to believe that all of the minute biological details are relevant. If you’re a functionalist you can work at a more abstract level. From the 12 years of research I did in this space (both CS and Bio) I can tell you for certain that we can learn a lot more from a functionalist approach.

And the reason we “can’t understand” 1000 layer deep neural networks is again because we don’t have a good theory of these types of computational systems. For instance I can tell you that most of the interactions w_ijk are completely spurious and if you cull the network you actually end up with a circuit that you can understand in the same way any EE undergrad can identify a 3 bit adder from the circuit topology.

[ Parent | on HN ]

Re:Why do CNNs generalize so poorly to small image transform... by SubiculumCode on Sunday June 10, 2018 @04:22PM (#17279976)

If there is one area of neuroscience where there has been a multi-level approach to understanding brain function it has been the study of the hippocampus. You will find detailed computational and simulation models built around detailed knowledge of its circuitry and a wealth of knowledge bridging behavior to hippocampal subregions, subcircuits, in both human and animal models. To me it is one of the most exciting areas of neuroscience because you can build up disting ideas about the functions of circuits based on te computational properties its structure could allow. Furthermore the hippocampus sits atop major networks where high level concepts and percepts are bound to create arbitrary relations for generalization and for one-shot learning.

1.http://science.sciencemag.org/content/315/5814/961?casa_toke... 2. https://onlinelibrary.wiley.com/doi/full/10.1002/syn.21972 3. https://www.sciencedirect.com/science/article/pii/S000689931... 4. http://onlinelibrary.wiley.com/doi/10.1002/hipo.22365/full 5. https://www.sciencedirect.com/science/article/pii/S092523121... 6. https://www.sciencedirect.com/science/article/pii/S089662731... 7. https://ieeexplore.ieee.org/abstract/document/7124441/ 8. https://www.sciencedirect.com/science/article/pii/S107474271... 9 http://science.sciencemag.org/content/353/6304/1117?casa_tok...

[ Parent | on HN ]