I find this argument to be an unsatisfactory against "algorithmic consciousness". There have been automated proofs of Godel's theorem for a long while [0]. I don't mean this to be evidence on the contrary, but Penrose seems to ignore the fact that a computer can realize multiple axiomatic systems, and use them to make statements like Godel's Theorem(s). Godel's Theorem(s) are often taken out of context for philosophical purposes, for better or for worse, and it's important to remember that Godel's Theorem(s) relies on a meta-language (ZFC) to make statements about PA.

For a much more intuitive explanation as to why consciousness is not algorithmic, I recommend "The Neural Basis of Free Will" by Tse. His argument is that neurons and neuronal circuits (and more) harness randomness to provide inputs to "criterial detectors" which are satisfied when the right combinations of inputs (spatiotemporal patterns) arrive at the detector at the right time. This can't be algorithmic, because of the requisite noise in the inputs and because the brain realizes true parallel processing. As a further note, he posits that free will is realized in the resetting of the input weights, so "current" actions set up the criteria for future actions avoiding the issue of causa sui in free will.

[0] - https://arxiv.org/pdf/cs/0505034.pdf

[1] - https://mitpress.mit.edu/books/neural-basis-free-will

> This can't be algorithmic, because of the requisite noise in the inputs and because the brain realizes true parallel processing.

A simulation of a brain or some set of neurons is an algorithm, as are algorithms that use nondeterminism or that simulate parallelism.

Is the claim by Penrose not that humans can possibly intuit theorems which are true but unprovable (Gödel's first incompleteness theorem) e.g. the Riemann Hypothesis, and thus how could an algorithmic process ever achieve such intuition?

Perhaps this is a Turing test for consciousness. "I can't prove this, but I've been thinking about these theorems [inserts true but unprovable list of theorems] and I think they're true".

>Is the claim by Penrose not that humans can possibly intuit theorems which are true but unprovable (Gödel's first incompleteness theorem) e.g. the Riemann Hypothesis, and thus how could an algorithmic process ever achieve such intuition?

A "true but unprovable" Goedel statement is true in standard models of arithmetic but untrue in certain nonstandard models. The "incompleteness" is syntactic, not semantic. The real and complex numbers, AFAIK, only have one model, up to isomorphism.

And sometimes statements are difficult to prove because they're actually independent of the foundational system. Or because a counterexample exists somewhere.

"This problem is unresolved, therefore it's a Goedel Statement within our current foundational system" is extraordinarily unlikely. For one thing, that would imply that we could figure out the axiom we're missing and pass to the stronger system capable of resolving the conjecture straightaway, or that starting from some stronger foundation like homotopy type theory would resolve the conjecture right-away. Most unresolved conjectures are not unresolved for lack of proof-theoretic strength in our foundations.

Humans also intuit a whole lot of theorems (loosely speaking) which are false. There is nothing prohibiting an algorithmic process from generating statements which are variously (and, from the POV of the algorithm, indistinguishably) true but unprovable, true and provable, and false.

How do you actually execute the test though?

A test requires that you be able to produce an artifact which is witness to the results -- what could you produce as the result of the test to distinguish between a true, but unprovable theorem and a nearly true theorem with a single (very, very, VERY) large counter-example?

>This can't be algorithmic, because of the requisite noise in the inputs and because the brain realizes true parallel processing.

A lazily evaluated, natively parallelized stochastic lambda calculus with Church-like query operators is still merely probabilistic-Turing-complete.

Arrrrrgh now you've got my CS401 knowledge screaming that someone is wrong on the internet.

That sounds about as convincing as saying "If you find a watch in the woods, you will surely never think that it happened to be created through a natural process; a living organism is much more complicated, so it must have been intelligently designed."

This argument has a name. I've seen it presented before and it is rather good. A gross simplification: a computer could never "see" the truth of Godel's Incompleteness Theorems, but we humans can; therefore, we cannot be computers.

It's not a good argument at all. It's circular, or a tautology depending on how you define things. You're using different words, but what you start with is the same as what you end up with. You assert that humans are different from computers: computers can't "see" something that humans can. And you use that to "prove" that humans cannot be computers. Of course. If you start by asserting humans are computers plus some magic, then of course humans cannot be mere computers.

You could write a computer program to generate random propositions, and test those logically against known truths. After enough testing, the program could assert that propositions not disproven are true. Voila! You have a computer program that has "arrived" at truths outside of a logical system. Some of those truths will be wrong, and some will be right, much like humans and their "truths" that are not based on logical reasoning.

If you've never encountered a human claim to "see" the truth of something that turns out to be false, well, it happens... a lot.

It's by Roger Penrose: https://en.wikipedia.org/wiki/The_Emperor's_New_Mind

Actually Penrose has the idea/argument from Lucas, which has been written in 1959 and published in 1961 : https://www.jstor.org/stable/3749270

Interesting! http://www.iep.utm.edu/lp-argue

Ah, right. It's called the Lucas-Penrose argument.

I'm not sure what you mean by "see", but there are computer verified proofs of Gödel's theorems: https://en.wikipedia.org/wiki/Gödel%27s_incompleteness_theor....

It's quite a stretch to say that we find truths that are beyond every possible formal system, rather than those that we've simply managed to encode in computers.