You extrapolate the trend from one data point. There is no indication that we have some matyoshka type hierarchy of intelligence. It could well be binary, either intelligent or not. No reason so far to think otherwise.
> You can't beat chess in one move because mathematics does not allow it.
So what superintelligence can do that "ordinary" intelligence in right quantities couldn't? Something specific beyond theoretical grasp of "just" intelligent machine?
Conceive of answers to this question that we can't. Do you really believe the human imagination captures all of possibility? Why?
Would be nice to hear some specific arguments against instead of Penrose-like quantum handwaving and pet analogies.
Well yes, because we are already generally intelligent. The problem is not to have the right hypothesis space built into our wetware, but to locate correct (action-guidingly veridical) hypotheses within the existing hypothesis space based upon sense-data.
If the AI runs on a normal computer, it is also Turing-complete and not beyond.
So they have the same limits
Compared to every other life form on Earth you are super intelligence. This super intelligence thing has already happened once. Of course this spread of super intelligence has wiped out most large mammals on this planet, 10s of thousands of other species, most of the fish in the ocean, and is adding gigatons of carbon to the atmosphere.
You can't beat chess in one move because mathematics does not allow it. Unless this said ASI develops bending 4D spacetime (in which you could beat chess in one move with some new rules) then the game simply does not have a piece that moves that way. That said ASI can be just a little smarter than us, and put us to extinction like we did with the Neanderthals.