I think your position couples intelligence with consciousness a bit too tightly. I can imagine a goldfish is conscious but not intelligent and I can imagine a very powerful supercomputer to be intelligent but not conscious.

The prerequisite to change your position can be satisfied by having a computer randomly generate a question, which I don't think would be an example of consciousness or intelligence. Furthermore even as a human (and one who is hopefully intelligent), I would not go so far as to say that I'm not programmed. Almost all of my opinions are programmed, the language I speak didn't just fall from out of the sky but was taught to me, my preferences are almost certainly due to programming and I am certain if I grew up in North Korea they'd be different.

All this to say that consciousness can be independent of intelligence, and both of them can be programmed.

As a lay-person, I was persuaded by Roger Penrose's argument that intelligence presupposes understanding, and that understanding is not computational. I had also finished reading a bunch of phenomenology before I read Penrose, so I was probably primed for that sort of an argument to be persuasive.

The difficulty there is, how do you understand things? "You" are mostly a chemical process, and I don't think there's much non-computational about chemistry. Penrose, as I understand him, answers "quantum mechanics", which I think is (a) kicking the can down the road rather than answering the question, and (b) problematic in its own right---the experiments supporting Bell's theorem seem to imply that QM and thus "understanding" are inherently random, right?

I actually do agree with you while still finding Penrose persuasive. I don't know how earnestly he adheres to the "quantum perturbations in nanotubules" (if I've gotten the incantation right) for which he credits someone else as a good try at a solution, and I have the impression he holds it as an open question.

I think the appeal (or interest, to me) from the phenomenology angle, is that we understand things by being in the world, having needs and desires, and some agency to obtain them. We understand a hammer because we can imagine a need to drive a spike, we can see that it can be held in our hands and be swung to drive the spike, that we have hands and a body, and that we occupy space.

All of that made me wonder if simply executing a program is sufficient for emergence of intelligence. This raises the naive question of whether adding enough sensors and actuators to the program would be necessary, or have I merely added something unnecessary to the problem? Anyway, that's the kernel of doubt that I have that a computer program alone is sufficient for the emergence of an intelligence, that is, an agent with the capacity to understand.

But why think understanding is not computational? It certainly enables a lot of computational behaviors, its effects can be modeled by computation. What power does understanding endow a system that a priori is beyond computation to capture?

'Consciousness' is just a synonym for self-aware. Goldfish are not self-aware.

How do you know?

Another fish, the cleaner wrasse, can apparently recognise itself in the mirror, hinting at some for of self-awareness.

https://www.nationalgeographic.com/animals/article/fish-clea...

A being is conscious if and only if it feels like something to be that being. What you're referring to is self-consciousness. Goldfish are most surely conscious but not self-conscious.

Stating that I can imagine X to be Y is not a statement that X is Y, only that if X is Y then nothing changes about my model of either X or Y.

That said, I'm not able to verify from a brief search for consciousness and self-awareness that the two are synonyms. The two seem to be related but are treated differently from one another. Furthermore it's not even clear whether goldfish are conscious or self-aware. Seems like it's an open question.

> How do consciousness and intention emerge from materials that are not alive and that have neither consciousness nor intention?

I think the question shoud first be not about "how", but rather - "does consciousness emerge from the materials".

Most cultures in the past have maintained ideas about the duality between spirit and matter. Nowadays we are so adept at manipulating matter we always explicitly assume that everything must be material in one way or another.

But here is this consciousness thing that does not give in. And there is no angle to it where it can ever give in. If somebody creates a machine and claims it to be conscious nobody can test if that is so. And further - nobody can even test wheather the inventor is conscious himself.

*Edited because I can't type*

I do not understand this argument.

I'm going use the word magic for whatever you're calling this non-"materialistic" spiritual explanation. I'm not trying to be a jackass, that's truly the best word I can find for it. I don't think magic is a bad thing, I just don't think you should summon magic until you need to.

Magic is not an explanation, it's a non-explanation. It's non-falsifiable (blah-bity Popper, yeah yeah).

Our adeptness at manipulating matter is not the origin of wanting (at least mine) to find a 'materialistic' explanation for something. It's the desire to have a predictive (read, useful) model of that thing, even if the model is not complete. Once you poke deep enough at any model you will find that there is some hand-wavy "magic" at the bottom. That is, there is some part that we don't know yet, either because we can't do the math, because we can't measure due to difficulty of experiment, or because of fundamental bounds where maybe you might be justified in starting to invoke some magic.

Despite best selling books about some quantum mysticism bullshit, I have yet to see an inkling of evidence for invoking magic for consciousness. From a physics perspective, biological systems are freaking hard, we have no reason to expect them to crack easily. And that's without us (physicists) understanding nearly enough about biology!

I don't think this line of dismissal necessarily fits. If we ended up having to expand physics to include some new (today seen as mystical) property of the universe, then you are essentially wrong here. With QM (which I doubt is related to consciousness) we had to add unmeasurable quantum phase to everything and accept certain previously-"nonphysical" new properties of systems. Similarly with other revolutionaly theories.

Magic can be disproven by explaining the phemonen with a conventional physical theory.

I don't think I follow? What am I dismissing?

I don't quite understand QM analogy. QM arose to explain new observation from more detailed measurements. Consciousness and intelligence and all things brain have been (to an extent) known 'properties' and difficult to explain (from first-principles) for quite some time. The more we probe, we answer some questions, and in the process reveal some more detail, leading to more detailed questions.

My qualm is that nowhere along this trajectory have I ever seen any need to invoke magic. It's a complex, many-body system with lots of noise, many relevant interacting length and time-scales that are difficult to cleanly separate, model, observe and probe. It's hard, but it seems we're making progress. Maybe it will take CERN, LIGO or ISS scale endeavors, I don't know.

It's not all similar, IMO, to how QM arose.

But I'm not a neuroscientist nor a historian of science.

What progress is there towards the "hard problem" of consciousness? Even a oversimplified approximation akin to the Rutherford atom? Research in the area is considered fringe anyway, as no one has any hope a researcher will get anywhere.

QM required that we accept a new phase property of matter which cannot be observed even in principle, and which causes counter-intuitive experimental outcomes that require contradictory physical explanations. This remains unsolved, physics simple sucked it up and moved on using it. The Copenhagen Interpretation essentially says some questions are not "askable" (e.g., what is the state of an unobserved electron), and observation instantantaneously changes vast-universe-wide systems via collapse of the wavefunction (pretty magical in the opinion of many). The many-worlds interpretation is even kookier, though much more intuitive, and is gaining popularity lately. If parallel universes and "spooky action ata distance" (quoting Einstein) is not magic to you then what does it take?

> I do not understand this argument.

The unfortunate thing is that, I think, I understand your side. But something happened, I read some book or heard some talk that placed a seed of doubt in my mind. At first it was just a seed and I oscillated back and forth between being 100% materialist and between entertaining the proposition that there is something beyond matter in this world. But as time went on I began to shift towards the "non material" interpretation for consciousness more and more.

It's not an explanation, I agree. But there is nothing to explain. Me claiming that consciousness might not arise from matter is not an attempt to explain it. I just see no way how it can be investigated in material terms. If you see a person on the street there is no way you can tell if he or she is conscious. And I don't see a possibility of there ever being a way.

Sadly, I don't know what started this doubt in me, so I cannot share it with you. You brought physicists for some reason, and I know a few, like Schrodinger, who thought about consciousness and came to the same conclusion. Here is his quote: "Consciousness cannot be accounted for in physical terms. For consciousness is absolutely fundamental. It cannot be accounted for in terms of anything else." I haven't read about Max Plank, but heard he had similar views.

If I had to guess my starting point was a book called "the problems of philosophy" by Bertnard Russell [1]. He tries to answer the question "are there any statements to which all reasonable men would agree". And one of the conclusions the book comes to is that you cannot claim anything as objective without assumptions, and that the most objective thing is your subjective experience.

For example if you saw a cat, turned your head, and then looked back at the cat - was the cat there when you were not looking or was it gone? In the book Russell convincingly demonstrates that if someone maintains the cat was gone when you were not looking - you cannot prove logically to that person that he is wrong. In other words - you would not be able to start with his worldview and lead him to contradictions. Hence his "theory" about cat disappearing cannot be disproven without assumptions about how material objects behave.

Not sure if this is helpful, but I wanted to reply.

[1]: https://www.gutenberg.org/files/5827/5827-h/5827-h.htm

Thanks for the thoughtful reply. I'm leaving this mostly to acknowledge it and mark so I can come back later with a more thorough response.

I don't think we strictly disagree.

For one, I agree that, fundamentally, as the observers, everything is "filtered" through us whether we like it or not. So there's no getting around some notion of 'subjectivity' or 'observer' bias, at least not that I know about. But that is not at all exclusive to an improved (predictive, quantitative, model-based) understanding of the brain and consciousness. Look how much progress we've made!

BTW I'm on my phone and forget why we're talking specifically of consciousness and not AI? Is that my fault? My bad.

Oh, I brought physicists so my perspective was clear and you could understand my bias and my ignorance of actual neuroscience and CS and biology.

You say there is nothing to explain, but there is something that has not been explained (by anyone), which is how minds work. If you are claiming that they cannot possibly be explained, that is just your intuition. You are, of course, entitled to your opinions, jus as much as those whose intuitions lean the other way, but there is no particular reason for us to regard anyone's intuitions on the matter to be an indicator of which way reality lies.

The observations of Russell's that you mention - which, I believe, were considerably influenced by his working with Wittgenstein - seem to be a critique of analytical philosophy's approach to learning fundamental truths about the universe. "Hold my beer", said his scientific contemporaries, "we're going to put these worries aside and see what we can learn anyway."

Taking an absolutist view that empiricism is the litmus test for all truth is similarly magical thinking, in my opinion.

Asserting “something that isn’t falsifiable isn’t real” is just as arbitrary as asserting that there exist phenomena that are beyond the absolute boundaries of our capacity to perceive them (that is, no amount of technical innovation will allow us to observe such phenomena). The latter statement, though, seems much more reasonable of an assumption.

My "blah-Popper" comment was a joke/self-deprecating to my argue. I agree that non-falsifiability isn't strictly sufficient grounds to dismiss something, but it is a curious smell.

Edit- crap, I'm pretty new at actually participating in HN comments, I don't think I followed chain properly

I would tend to agree with your claim "the latter statement, though, seems much more reasonable of an assumption", but that does not give any particular reason to think that minds are examples of such phenomena.

Certainly not, my only goal is to assert that it is a perfectly reasonable hypothesis.

Yes, the uncertainty of what can be known and the arbitrariness of empiricism make a wonderfully powerful hypothesis generator, allowing one to hypothesize against not only unverified hypotheses, but also against any claim that any given hypothesis has been empirically verified. One can, for example, employ it to say that it is a perfectly reasonable hypothesis that the apparent global pandemic is not being caused by a virus, or that homo sapiens did not evolve from other species.

I mean that’s getting into the point of being unnecessarily facetious.

If something is true from an empirical perspective that is one thing. It is another thing to say that even though we have been unable to form a conclusion from an empirical point of view, the ultimate ability to do so is given, and that all hypothesis that cannot be tested in such manner must be completely disregarded, even if they make sense.

My post was influenced by your comment about absolute empiricism being magical thinking. Now, you did qualify ‘empiricism’ with ‘absolute’, but the author of the post to which you were replying seemed to be adopting a pragmatic form of empiricism, much like the one you have adopted in the first part of your last post. It seems to me that bringing up ‘absolute empricism’ here seems to be something of a straw man (though maybe the original post was edited before I saw it.)

FWIW, I completely agree with that person’s view that saying “Well, it’s not physical” to the question of what a mind is and how it works, is a non-answer that avoids the question. Materialism has not answered that question ether, but it is a double standard to deprecate only materialism on that basis, as the author of this thread’s root post did.

The philosophy of mind seems to me to be hobbled by its fascination with zombie arguments, which are rather cleverly constructed to persuade one that a philosopher’s metaphysical speculation is a stronger guide to how the universe is than any amount of empirical physical evidence.

It seems to me that consciousness emerges when things arranged in some particular way are acted upon some principle, which is true even for simple behavior, like a rock falling to the ground.

If you put a sail on a boat, it will catch the wind and move across the water. It's not the sail alone that moves the boat, nor the wind, but their interplay. A sail without wind doesn't get moved, and a wind without anything to catch it doesn't move anything.

May be a bit of an unintuitive conclusion, but I think may be that we are catching the consciousness like a sail catches the wind, and this is the same "wind" that we call the forces of nature, that we only can perceive as patterns in how things around us appear to change.

I think neuroscience could in principle show that phenomenal consciousness is an illusion, by giving a full account in terms of the brain of why it seems to us that we're conscious. Whether it will is another question.

There'd still be a "me" there who would be having that illusion.

Regardless of whether we know exactly how it works or not, I have a subjective experience.

You seem to be adopting a double standard here. It is true that materialists cannot explain how minds work, but dualists can do no better, and, furthermore, they cannot point to any remotely similar phenomenon that is non-physical. Your claim that "there is no angle to it where it can ever give in" is just as much a speculative intuition as the claim that minds are the result of physical processes.

The position you take in your last paragraph is one of solipsistic skepticism. Do you approach every epistemic issue with the same level of skepticism, or do you only adopt it with regard to the proposition that minds are the result of physical processes?

We do know that if we add chemicals, say we drink some amount of alcohol, that can alter our consciousness. That wouldn't happen if it were completely nonphysical.

> As an AI professor, I've always held that machines are NOT intelligent (I am prepared to change my position on the day my computer asks me anything surprising that I didn't program it to).

While granting you your seemingly arbitrary metric for 'intelligence', one is forced to wonder to what extent AI systems of today are even given a means by which to ask such questions if they did, in fact, have them. Take AlphaGo, for instance, which can consistently beat the greatest living grand masters of the game. Does it have any means of interaction by which it could pose an unexpected question?

One can reasonably ask whether the techniques that produced AlphaGo could produce a program that asks unexpected (yet pertinent) questions. I'm not sure, but I think we can say it has not been demonstrated yet.

Is it the case that playing the games are effectively a way of asking and answering questions, specifically ‘do I win if I do this’?

That wouldn't qualify as 'unexpected' in any way. The system's ability to express any questions it might have is severely curtailed by the limitations of its inputs and outputs.

And yet the go games had some very unexpected and creative plays, so if the inputs and outputs were language based are we simply moving the goalposts for what is considered 'unexpected'?

It is more a case of moving the goalposts back where they belong, I believe. The original moving of the goalposts was in calling any 20th-century technology AI.

I’m very interested in asking a question to you; what is intelligence? When deep blue beat a grandmaster it was just memory and algorithms. When GPT3 writes creatively from prompts and answers it’s just a transformer with a lot of parameters. When we use deep learning it’s just gradient descent. When we we have object recognition it’s just matrixes. When we use text prediction, it’s not intelligent, it’s just a markov chain.

The goalposts for intelligence keep changing. What exactly is intelligence?

Ah, but the question remains: Are you intelligent? :-)

How do consciousness and intention emerge from materials that are alive and that have neither consciousness nor intention?

How does life emerge from materials that are not alive?

It is still possible to believe all of current scientific knowledge and also be a vitalist without contradiction, but few people aware of the state of current knowledge hold that artificial life is utterly implausible.

Einstein famously held that certain experimental outcomes could only be explained by local hidden variables, but we have since found, to the contrary, that they actually rule out local hidden variables as an explanation.

Statements of incredulity are not, by themselves, evidence against a proposition.

Thank you. This all seems very simple to me: So-called AI is mostly not different from books, movies, videogames etc. No, they don't "think" and they are not "intelligent."

But that in no way precludes that it an instance of it being mind-blowing and world-changing. Books, etc. do that, too.

Can a book drive a car by attaching some sensors to it?

AI models definitely have intelligent aspects to them since they perform tasks we can't write algorithms for manually in a straightforward way.

But a movie or a book is nothing like an AI model, they don't process information, they are static. AI models can react to numerous inputs in various ways.

I mean, a light switch reacts to my input by turning on.

If the measure is "reacting to numerous inputs in various ways," the Bible's definitely more intelligent than any "AI." It already figured out how to control humans.

(Like, I'm joking but I'm not? It really does speak to my opinion that at the end of the day "what humans do" is the thing that matters. People will get inspiration and ideas from both AI's and books. And people will use and misuse machines and computers, but that's about it.)