Deep learning is like fish, there are a lot of types of fish. Some will eat you and some you eat.

My understanding of deep learning is that it is a methodology, not a technique.

Deep learning is all the very many models that use layers of neural networks. You can tweak it a lot, but there comes a point where you're not working with neural networks anymore. The human brain has reached that point.

I wonder sometimes how tiny brains work. Take butterflies: they can flit among flowers on a bush or in a small area on the ground; how do they know how to move around so they fly to a nearby flower, then stop and insert their proboscis into the nectar-bearing region? I guess there is no real learning going on--the behavior is probably innate--but still, it seems miraculous.

I agree, it is miraculous. the fact that dna is ultimately storing all the info for specifying neural circuits that robustly support such complex innate behaviors (often with very little post development tuning / learning) , that to me is mind-blowing. And butterfly behavior is one thing, but what about innate detection of predators in some visual systems? How do you encode a snake detector in dna?

I believe they call that Hebbian Learning in neuroscience.

Not really, hebbian learning is also learning from experience. Butterfly behavior is probably mostly hardwired.

> there comes a point where you're not working with neural networks anymore. The human brain has reached that point

No offense, just startled, but that’s the most self-contradictory thing I’ve ever heard.

The human brain is by definition a neural network…

"Neural Network", tragically, is a term that refers to a specific kind of model inspired by a sort of 70s understanding of human neurons. Deep learning refers to those models. The human brain is made of neurons, and by God do they network, but the term has been co-opted so it is not a neural network. A particular region or set of neurons that do one thing might be called a neural circuit, and the whole thing can be called the connectome (though that's sort of debatable).

Maybe it’s time for “academic justice” to reclaim the term.

I'd love nothing more, believe me. But even if neural networks were taken to unambiguously refer to the neurons of animals, deep learning would still refer to the computational model, whatever its new name would be.

Technically, Artificial Neural Networks are what people mean when they say Neural Network, right? Biological Neural Network probably ought to be the default meaning of Neural Network, given the implicit homage to biological neurons.

I always wondered why people stopped referring to ANN and just went with neural network. Perhaps some anti-science bias or something like that.