Nope. I don't think anyone thinks the brain is an N layer deep network. There are many networks, and they cross-couple inputs and outputs from various subnetworks into others. There are also hormonal regulatory systems with feedback mechanisms that interact with these networks both is inputs and outputs. The brain is hugely more complicated and messy than anything that, say, GPT-3 is doing.

I'd also add that the subconscious parts of the brain seem to function quite a bit differently from the conscious parts. It's hard to reason about something like "why does my eye see how it does" when those parts aren't consciously accessible.

So Parallel Distributed Processing, it is?

Apparently deep learning has been beat-to-death as a word, but the way deep learning was explained to me, it is an approach not a specification.

Yeah, if you dilute the meaning down to “feedback loop”, then deep learning and the brain work the same way.

I’m not sure that it is correct to limit deep learning to just a feedback loop, but cybernetics (if the thermostat variety, not the terminator variety) might be the right word.

Brains are

a) far far more recurrent than any practical deep learning system

b) using something other than backprop, and probably that something is much more efficient

Deep learning is like fish, there are a lot of types of fish. Some will eat you and some you eat.

My understanding of deep learning is that it is a methodology, not a technique.

Deep learning is all the very many models that use layers of neural networks. You can tweak it a lot, but there comes a point where you're not working with neural networks anymore. The human brain has reached that point.

I wonder sometimes how tiny brains work. Take butterflies: they can flit among flowers on a bush or in a small area on the ground; how do they know how to move around so they fly to a nearby flower, then stop and insert their proboscis into the nectar-bearing region? I guess there is no real learning going on--the behavior is probably innate--but still, it seems miraculous.

I agree, it is miraculous. the fact that dna is ultimately storing all the info for specifying neural circuits that robustly support such complex innate behaviors (often with very little post development tuning / learning) , that to me is mind-blowing. And butterfly behavior is one thing, but what about innate detection of predators in some visual systems? How do you encode a snake detector in dna?

I believe they call that Hebbian Learning in neuroscience.

Not really, hebbian learning is also learning from experience. Butterfly behavior is probably mostly hardwired.

> there comes a point where you're not working with neural networks anymore. The human brain has reached that point

No offense, just startled, but that’s the most self-contradictory thing I’ve ever heard.

The human brain is by definition a neural network…

"Neural Network", tragically, is a term that refers to a specific kind of model inspired by a sort of 70s understanding of human neurons. Deep learning refers to those models. The human brain is made of neurons, and by God do they network, but the term has been co-opted so it is not a neural network. A particular region or set of neurons that do one thing might be called a neural circuit, and the whole thing can be called the connectome (though that's sort of debatable).

Maybe it’s time for “academic justice” to reclaim the term.

I'd love nothing more, believe me. But even if neural networks were taken to unambiguously refer to the neurons of animals, deep learning would still refer to the computational model, whatever its new name would be.

Technically, Artificial Neural Networks are what people mean when they say Neural Network, right? Biological Neural Network probably ought to be the default meaning of Neural Network, given the implicit homage to biological neurons.

I always wondered why people stopped referring to ANN and just went with neural network. Perhaps some anti-science bias or something like that.

> If deep learning isn’t enough, then brains aren’t enough, right?

No, not right. Humans need other humans to be smart, we need culture, technology and science. We're not that smart - just 700 years ago we were dying of the plague without even knowing the germ theory of disease, even with our lives on the line we couldn't solve it. We're only INCREMENTALLY smart over our current cultural level.

How do you explain non-human intelligence and tool usage, where there is no known culture?

Actually, tool usage in primates is a form of culture (protoculture). Those groups that have mastered it can be better of than those that didn't -- giving them an evolutionary advantage. The "tools" can be suprisingly complex, see for example:

https://web.archive.org/web/20110608070200/http://www.uic.ed...

What about birds?

https://www.wired.com/2011/01/new-crow-tools/

That is pretty interesting -- I didn't knew that. I would classify this behaviour also as protocultural.

What about swarm intelligence of ants and bees?

Protoculture as well?

They can still individually learn tool usage by imitation, random exploration and reinforcement. But without culture they can't build on it.

Human supremacist world-view. Sadly incredibly common, once you drill down.

It's culture-supremacist actually. Information needs to copy itself into the future, that's the essence of both life (genes) and culture (memes).

Culture supremacy is still just bio-supremacy. Memes don’t exist except through genes.

Arguing that memes exist outside of genes is like arguing that the Platonic solids are real.