I would not be so reductionist. Intelligence doesn't seem to be an universal thing, even IQ (a human invented metric) is measured in terms of some statistics. If you have an IQ of ~60 you have intelligence but a completely different one from an IQ >85.
> But humans are intelligent, human languages have a universal grammar and can be indefinitely extended with new words. I think this puts us in the intelligence species club, and all species in that club can all discuss any idea.
Humans have different intelligences. You can be intelligent (per the human intelligence definition) but a math ignorant. Again, this implies intelligence as we know it is not an universal thing at higher levels: not all people can have a physics Ph.D. as not all people could be a good artist where good techniques are recognizable, same for music, etc.
Once you have "enough" intelligence to have a complex language, like we do, I'm claiming you are in the club of intelligent species, and all species in that club can communicate ideas with each other. Even if the way they natively think is quite different.
The AIs might spit out entire fields of knowledge, and it might take humans decades of study to understand it all. And no single human might actually understand it all at the same time. But that's how very advanced fields of study already are.
But the "cockroach" slur implies AIs would be in this other stratosphere having endless discussions that we cannot remotely grok. My guess is that won't happen. Because if the AI were to say "I cannot explain this to you" I'd take that as evidence it wasn't all that intelligent after all.
There are already math proofs made by humans on this very day that are hundreds upon hundreds of pages of lemmas that are highly advanced and building on other advanced results. Understanding such a proof is an undertaking that literally takes years. An AI might end up doing it in minutes. But what an AI could cook up in years could take a human... several lifetimes to understand.
As another example, take the design and fabrication of a modern microprocessor. There are so many layers of complexity involved, I would bet that no single person on this planet has all the required knowledge end-to-end needed to manufacture it.
As soon as the complexity of an AI's knowledge reaches a certain point, it essentially becomes unteachable in any reasonable amount of time. Perhaps smaller sub-parts could be distilled and taught, but I think it's naive to assume all knowledge is able to be sliced and diced to human-bite-sized chunks.
For example, if the AIs could design a complicated building, using our level of technology, they could explain to us how to build that building. And we could build it. Whereas if we come up with a better cockroach-house design, we cannot communicate it to the cockroaches, we simply cannot give them the information. So the AI->us is a very different relationship from us->cockroach.
This doesn't preclude that there might be some things the AI cannot explain to us. Only that there will be many things (infinite in fact) which they can explain to us.
I always thought you could ask GPT to illustrate the steps it took to arrive at the answer. I mean it can take your through the process it went through to arrive at the answer. Its as close you get to an explanation.
What if someone tries really hard for a long time and can’t learn a field? Do they fail the intelligence test, or does their teacher?
I submit that not everything can be hierarchically decomposed in a way that's useful - i.e. any "abstraction" you try to force on it is more leaky than non-leaky; in that it doesn't simplify its interactions with other chunks. You might say it's the wrong abstraction - but there's no guarantee there is a right abstraction. Some things are just complex. (This is hypothetical, since I don't think we can conceive of any concepts we can't understand.)
An AI could have an arbitrarily large working memory.
Note: I'm talking about intuitive understanding. We could use it mechanically, just never "get it", cowering before icons, being the one in Searle's Chinese Room https://wikipedia.org/wiki/Chinese_room
And I think this is all very unlike how we are currently impacting the lives of cockroaches with our insights about quantum computing. Thus, it's not a good analogy.
Then try explain quicksort to them. Obvious waste of time.
They wouldn't be much threat in a zero sum strategic interaction either.
My claim is just what AIs can teach competent smart adults is many many orders of magnitude more than what humans can teach competent smart cockroaches. Thus Hofstadter's analogy is not a good one.
Are dogs, or pigs, or whales, part of the intelligence club? They are clearly intelligent beings with problem-solving skills. We won't be teaching them basic calculus any time soon.
Intelligence might be a spectrum, but powerful generative language is a step function: you have it or you don't. If you have it, then higher intelligences can communicate complex thoughts with you, if you don't they can't. We have it, so we are in the club, we are not cockroaches.
there are many humans who could study mathematics for a lifetime and not be able to comprehend the current best knowledge we possess. i'm one of them. maybe it takes 2 lifetimes. or many more.
a human-level AI operating at machine pace would learn much more than could ever be taught to a human. our powerful generative language capabilities wouldn't matter - it's far beyond our bandwidth. especially so for a superhuman-level AI.
The AIs will deliver to us truly massive quantities of information, every minute, until the end of time, much of it civilization-changing. Thus the AIs relationship to us will thus be nothing like our relationship to cockroaches, where we essentially cannot tell them anything, not even the time or the day of the week, let alone the contents of Wikipedia.
I think Hofstadter is having an emotional reaction to AI. He says so as much. And it'a a common one, it's the woe is me phase. But I think he's totally wrong about the analogy. I'm 100% sure we will not feel like cockroaches when AI is in full swing, not in the slightest.
> Imagine an AI that could fundamentally alter its own sensory perception and cognitive framework at will. It could “design” senses that have no human equivalent, enabling it to interface with data and phenomena in entirely novel ways.
Let’s consider data from a global telecommunication network. Humans interface with this data through screens, text, and graphics. We simplify and categorize it, so we can comprehend it. Now imagine that the AI “perceives” this data not as text on screens, but as a direct sensory input, like sight or hearing, but far more intricate and multidimensional.
The AI could develop senses to perceive abstract concepts directly. For instance, it might have a “sense” for the global economy’s state, feeling fluctuations in markets, workforce dynamics, or international trade as immediately and vividly as a human feels the warmth of the sun.
Simultaneously, it can adapt its cognition to process this vast and complex sensory input. It could rearrange its cognitive structures to optimize for different tasks, just as we might switch between different tools for different jobs.
At one moment, it might model its cognition to comprehend and predict the behaviors of billions of individuals based on their online data. The next moment, it might remodel itself to solve complex environmental problems by processing real-time data from every sensor on Earth.
In essence, the AI becomes a cognitive chameleon, continually reshaping its mind to interact with the universe in ways that are most effective and efficient. Its thoughts in these diverse cognitive states would likely be so specialized, so intricately tied to the vast sensory inputs and complex cognitive models it’s employing, that they are essentially impossible to translate into human language.
It’s be like trying to fit GPT-4 onto a floppy disk.
So yes there might be an infinite amount they cannot convey, but there is also an infinite amount they can convey. I guess it's half-glass-empty test if you are happy about the infinite you get, or are just sad about the infinite you don't get.
Infinities can be very constraining.
Now imagine being an AI and creating a human-readable library with 50M AI-written books for us to read. They could easily do that. And then create 50M more, again and again. And they could read every book we wrote. And forget books, humans and AI could have hundreds of millions of simultaneous real-time conversations between humans and AI, forever, on any topic.
So being a human in an AI worlds is nothing like being a cockroach in a human world. Sam Harris used the same analogy but said we were ants instead of cockroaches I've heard bacteria also. I think people trot out these bad analogies strictly because it sounds dramatic, and being dramatic seems like good way to get people's attention. Or else they just didn't think it through.
Human language is a Big Big Deal. It's a massive piece of cognitive technology. Any intelligent species with language is in the club and they can communicate with all other intelligent species -- even if those species have very different cognitive capabilities.
I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms of intelligence, even if they are much "smarter".
Cockroaches are not an intelligent species, so they cannot understand our thoughts. But humans are intelligent, human languages have a universal grammar and can be indefinitely extended with new words. I think this puts us in the intelligence species club, and all species in that club can all discuss any idea.
AI might eventually be able to think much quicker than us, to see patterns and make insights better and faster than us. But I don't think makes us cockroaches. I think if they are so smart, they are by definition smart enough to explain us any idea, and with effort we'll be able to understand it and contribute our own thoughts.