My fear is that we will ascribe too much human behaviour to that we see in and understand of our AI inventions, and that this will result in the dehumanisation of people.
So essentially my fear is what we justify doing to each other due to AI, rather than what "AGI" could do to us.
So while I'm sure there are a lot of custom tuned, problem specific hardware structures in our brain architecture, we do seem to learn how to actually use that hardware individually. As a result we seem to come up with a diverse range of different high level approaches.
That became popular in neural networks after the introduction of dropout regularization, which forced neurons to "co-adapt" and learn to do each others' jobs. Large, over-specified models also provide a natural setting for co-adaptation.
Improving neural networks by preventing co-adaptation of feature detectors https://arxiv.org/abs/1207.0580
I’ve been reading a pop neuroscience book called Incognito (2011).
In it, the author talks about how the brain is a group of competing sub-brains of many forms, and the brain might have several ways of doing the same thing (e.g. recognizing an object). The author also posited that the lack of AI progress back then was due to the fact that there are no constantly competing sub-brains. Our brains are always adjusting and trying new scenarios.
I was struck by how similar these brain observations were to recent developments in AI and LLMs.
The book is full of cool stories, even if some of them are now recognized as non-reproducible. I recommend!