Many animals are not generally intelligent.
Also many human subsystems are easy to trick. Look at eg optical illusions.
And, to follow your philosophical argument: given that the only thing that connects our cells to the cells of our ancestors is information, why couldn't those billions of years be also in our computers?
> And, to follow your philosophical argument: given that the only thing that connects our cells to the cells of our ancestors is information, why couldn't those billions of years be also in our computers?
Information is passed through the generations by DNA. Silicon doesn't have DNA.
Why? We might get taken out by a meteor tomorrow. I agree that, in principle, AGI is possible. But there's lots of things that are in principle possible that will never happen.
We have general intelligence but it took billions of years of evolution for us to get here. In some sense those billions of years of evolution are still with us, in every cell in our bodies. Maybe all of those years of learning are necessary for general intelligence. Trying to find a shortcut may yield impressive results in narrow circumstances but I doubt it will be generally intelligent in the way people and animals are.