I assume the goal is to achieve something useful. If AGI is impossible, then much more useful than these incremental steps is to figure out how to most effectively combine human cognition and computational systems.

> then much more useful than these incremental steps is to figure out how to most effectively combine human cognition and computational systems.

You don't know that

And you don't know AGI is possible. Both are options, and we should first do some kind of basic research to determine which is a better route before dumping billions of dollars in one direction. Or, at very least we should dump money in both directions, and tune towards which is getting better real world gains.

Obviously it's possible, we're having this conversation aren't we?

In the sense that the "Artificial" part of AGI means: created by humans, we do not know if it is possible.

I mean... what's so special about biology? How about I flip a few base pairs here and there, artificial enough yet? We already routinely genetically engineer mice and other model animals such that particular bits of their brain will either glow or fire in response to laser light, we have the technology today to put make little mice helmets that we can use to steer mice around - just need to do a bit more research to find out what particular bit of the (probably) hippocampus to stimulate. Is that artificial enough?

OK and how about just simulating the universe? There is a legitimate question about computability there - it does seem plausible that aspects of simulating the universe could be uncomputable.

Suppose that this is case - well it's still an open question as to whether or not this spells doom for the simulation route. The uncomputable bits are going to be some quantum this or that, and it's not at all clear that such low-level bits are fundamentally required for human-level intelligence; that the high-level process of intelligence is inseparable from the underlying processes which give rise to human intelligence.

Personally, I find it a highly unlikely that intelligence is inseparable/has no reduced model, for whatever my prognostication is worth.

BUT even if it is inseparable, there's still a strong argument to be made that you could construct AGI through means of so-called 'embodied computation', just like biology does.

Embodied computation is not AGI. For example, if it is an immaterial soul that is doing the non reducible processing, and we take the bit of matter that the soul communicates with the world through, we have not created AGI. We've only managed to entrap a beings soul in a machine.

If we can empirically quantify the behavior of an irreducible entity, which seems plausible, then the hypothesis is scientific.

I don't understand what you mean. We are having this conversation about AGI so AGI is possible? We can talk about 1 + 1 = 3, but that's not possible...

Or are you saying only AGIs can have conversations? That's begging the question.