For two reasons. First of all IQ is defined as a normal distribution with a mean of 100 and a sd15. So by definition only 0.1% of the population can have an IQ of above 145 and this fraction is rapidly diminishing with additional standard deviations.
Secondl: the definition matches reality reasonably well because intelligence is a polygenetic trait and many small factors add up to a normal distribution.[0] It is statistically unlikely that natural processes will yield an IQ500 human. You would have to engineer one. Which leads us back to the AGI concern.
Also, I'm still confused as to what you mean by : AGI concern. What's the concern? What's the big fear?
IQ is not a linear scale, all it tells you which quantile of the population one falls into. It does not describe the relative strengths between the quantiles in particular tasks.
To figure that out there are additional surveys that map IQ ranges to occupations and the tasks an individual is required to perform.
Could a theoretical computer scientist give more insight on this? In particular, what is the hardness of approximate solutions to most problems that could have strong impact in the real world?
[1] I did, in passing, find a similar idea discussed somewhere afterwards. If someone has a reference for that, I would be glad to have it.
I'll instead give you something to consider : * The totality of mathematics is not fully defined. As such, there are can be completely new mathematics that destroys previous conceptions especially as it relates to the limits of computability. Theoretical Limits often reflect one's limited scope of perception. Your limits or the limits someone theorized ages ago aren't necessarily my limits if I find a path beyond them.
Information theory and computational complexity theory are theories... Solving fundamental aspects of intelligence will invalidate a host of theories, create new mathematics, new algorithms, and new theories. Continuing to frame something as ground breaking as an understanding/implementation of intelligence in yester-years theoretical limits is flawed.
It might explain why humans are all close in intelligence to each other. Eg even with seven billion people we’ve never seen someone with a 500 iq.
Or that you can explain the work done by the smartest human to someone with average iq if you invested some time building up their knowledge.