> That also means that very big, complex artificial intelligences run on dry silicon will produce big, complex, unhuman-like minds.
Incidentally, this is exactly what the AI superintelligence people are terrified of, and what the paperclip problem is meant to illustrate. https://www.decisionproblem.com/paperclips/index2.html
Similarly, there is an extreme amount of pro-human arrogance in the article "At the core of the notion of a superhuman intelligence — particularly the view that this intelligence will keep improving itself — is the essential belief that intelligence has an infinite scale. I find no evidence for this.... So the question is, where is the limit of intelligence? We tend to believe that the limit is way beyond us, way “above” us, as we are “above” an ant. Setting aside the recurring problem of a single dimension, what evidence do we have that the limit is not us? Why can’t we be at the maximum?"
Humans are just a bunch of atoms that randomly learned to think over a few billion years. Given how fast we can improve tech via directed means, we should be able to replicate what nature has done in thousands, hundreds, or tens of years.