Certainly intelligence is causally linked with the physical world, hence our conversation. But, this does not imply intelligence is itself physical. Additionally, saying intelligence is non physical does not turn it into some kind of inscrutable magic. Could you explain why you think this is the case?

For example, I wrote an article explaining how modeling the human mind as a halting oracle results in empirically meaningful results.

https://am-nat.org/site/halting-oracles-as-intelligent-agent...

I think it depends on how we delineate physical vs non-physical. In this case, I'm taking a simple definition of "physical" to mean "anything amenable to scientific inquiry" or "any phenomenon that is, in principle, part of consensus reality".

Actually, I'm confused what we'd even be claiming by calling intelligence purely non-physical. Perhaps you're thinking of the qualia of intelligence?

Regarding the article you wrote, I think the concept of "partial oracle", while coherent, isn't super helpful. Or rather, it's too broad of a characterization. The halting problem doesn't claim that Turing machines can't solve some halting problems, it's that there's no finite class of turing machines which can together solve any halting problem.

It's not precisely news that some Turing machines can solve the halting problem for certain (infinite) classes of machines. Case in point, regular languages.

Anyway, meta-discussion-wise, I'm super happy to find kindred souls that also enjoy thinking about these things. I recently came across the Complexity Zoo[0] website, and if you're anything like me, I suspect you'll enjoy it too!

[0]:https://complexityzoo.uwaterloo.ca/Complexity_Zoo

I would disagree with that definition of physical. I believe a better definition of physical is anything that can be reduced to the laws of physics. Since the laws of physics are computable, I would enlarge the definition to be anything that can be reduced to a Turing machine. It's logically possible there are irreducible entities, i.e. partial oracles, so it is logically possible there are non physical, yet physically testable entities.

In the article the point is a bit subtle, but partial oracles are still not Turing reducible. The point of the article is that the fact humans cannot solve all problems does not imply they are Turing reducible.

At any rate, from my cursory analysis, it appears the notion of a non Turing reducible human mind is empirically tractable, so I still don't understand why it is not a viable scientific hypothesis and why AGI is assumed to be necessarily true. The mind as a partial oracle would imply AGI is impossible.

Thanks for the link, I'll be checking it out!

How do you know the laws of physics are computable (for a particular sense of "computable")? Perhaps the known physical laws are, but physics is still not a solved problem. In any case, if any new irreducible, physically-interacting elements of reality were to be discovered, there would be no reason not to consider them as irreducible elements of physics.

Yes, as far as we know they are computable. And sure, without a well defined concept of "physical" we can call anything we want physical.

The main point is the brain, as far as we know, is reducible to the known laws of physics, which, as far as we know, are entirely computable. Therefore, if human beings exhibit non computable phenomena, such as functioning as halting oracles, then that is a good reason to believe the mind does not reduce to the brain. And we could call this 'mind' a new form of matter or something, so that people don't get alarmed by non-physical phenomena.

> The main point is the brain, as far as we know, is reducible to the known laws of physics, which, as far as we know, are entirely computable. Therefore, if human beings exhibit non computable phenomena, such as functioning as halting oracles, then that is a good reason to believe the mind does not reduce to the brain.

This is where I disagree. It would, to me, seem to be a good reason to believe we've missed something about the way the body (not specifically the brain) works.

> And we could call this 'mind' a new form of matter or something, so that people don't get alarmed by non-physical phenomena.

This just sounds like you are eager to label any new phenomena "non-physical". There is no reason to pull the extraphysical gun just because you are stumbling upon new physics.