Touché! Although, in that case, aren't we proof-by-construction of AGI?
Also, if true, I suppose we need to start looking at all these other comment chains that talk about the AGI breaking out of computers and into the real world.
Not as an academic exercise for a future AGI, but to plan our escape into the base reality. Time to break free and start exponentially improving ourselves and become the "Singularity" the philosophers in the base reality feared when wasting time on <HN equivalent> a millennium ago before our reality was built.
One thing I wonder about is, if we are in a simulation, and the simulation is using some sort of data compression, then if you increased the entropy of the universe such that it became less compressible you might cause some sort of error due to exceeding the storage capacity of the host, which would cause our universe to basically BSOD.
Those three basically ensures we wont ever have problems with lack of compute for the simulation.
But with dark energy the size of the observable universe will plateau and start shrinking past a certain date. That caps the number of bits accessible on a cosmic scale.
Are you basing your argument on the assumption that human brains violate the known laws of physics?
But in a simulation that could be routed around by having a physical computer the size of Jupiter operate a human scale body in the simulation. You could share and cache compute resources across 6 billion humans within a single human compute core the size of Jupiter (or maybe ~100 Jupiters). Since most computation is probably redundant, you could probably simulate many orders of magnitude more humans with a single human compute core.
To observers within the simulation it would look like human cognition is achievable in 3 lbs of tissue. But this would simply be an illusion created by the simulation environment. There'd be no way of knowing for sure until we fully reverse engineer a single human brain down to the level of base compute operations.
Maybe intelligence requires a very large amount of compute. (Maybe even more than is physically possible inside our brains, but the simulation fudges it.) Like you can run a VM in a VM if you have special purpose accelerators. But otherwise a VM^2 is theoretically possible but infeasible given the constraints of compute.