Neurons have a lot going on, they send and receive signals through a multitude of mediums, not just neural impulses, and they're capable of plasticity when it comes to the connections they make between other neurons. Neurons also don't have simplistic activation functions, they're capable of doing a lot more with the information they receive and send. Also, gradient descent and back propogation don't take place in any part of the brain.
Through that lens, I see NNs as if they're like really complex and impressive Markov chain generators. They can produce results that look intelligent, but it's just statistical correlations, and not at all how the brain works.
What if this a more accurate representation:
"It's like the connection between dictionaries in real life and dictionaries in programming."
The neuron could be implementing NN theory in a way that is optimized for it's environment.
Without defining what's essential, I'm nervous to call the comparison insufficient. If a topological subset of neurons isn't good enough, what do we need in addition/instead? If we stuff NNs full of complicated (how complicated?) activation functions, does that new system do the trick? Or add...47 new "neuron" variants? Or swap the learning scheme from gradient descent to something fancier? (For that matter, do we even know what the brain's scheme is, and why GA/back prop isn't an acceptably extremely crude approximation of it?)
The brain is so unimaginably intricate. Our models are hilariously simple in contrast, of course. But what of those mismatches are differences in kind vs. differences in magnitude?
"All models are wrong. Some are useful"
-George E. P. BoxFunnily enough, that’s how I see brains. They start out as a few neurones basically implementing ‘hard wired’ logic, then some feedback loops form and next thing you know they’re asking “why am I here?”
These "just so" stories are attractive but it is quite important to realize a metaphor which is intuitive and you perceive as useful is nothing at all like the process for finding real scientific truth. There is also a lot of introspective value to modeling the world as being controlled by mysterious gods who are pleased or appalled at your behavior and that's why good and bad things happen. Perhaps useful for some people but nothing at all like truth.
When I said "I can't stop," I was referring more to this tendency to borrow models to explain unrelated systems. It's just a thing my brain wants to do and I can't help it (and again, I seem to convince myself that it's somehow accurate or useful even if, rationally, I'm quite sure it's not).
They keep telling me this and, yet, I can't stop doing it. The more I learn about neural networks, the more I feel like I understand my own brain (whether accurate or not). And conversely, the more I think about thinking, the better my theories about how I'd build ML-based system to solve specific problems (admittedly, most untested). Neural networks seem like too useful of a model to simply give up because they aren't completely accurate.
Of course, this is all just for personal use - mostly introspection. I wouldn't exactly do medical work based on the model.