News for hackers. Stuff that matters.
Re:Consciousness is not computation by hsn915 on Monday June 20, 2022 @03:03AM (#31807153)
Computation does not have anything essential to it that makes it different from other "atoms jumping around" other than it produces outputs that we find interesting.

There's no reason to expect an adder circuit to be aware that it's adding numbers. There's no reason to expect an implementation of AlphaGo to know that it's playing Go. This should be extendable without limitation.

There's no reason to expect that a certain type of circuit produces consciousness unless of you have a model of consciousness that allows for certain properties to cause it. But there's no such model.

The only thing there is is our brains. You just think that since the brain is doing computations and the brain is conscious, therefore consciousness is a computation.

Re:Consciousness is not computation by TheOtherHobbes on Monday June 20, 2022 @03:31AM (#31807308)
The argument seems to be that beyond a certain level of complexity you'll somehow automatically get consciousness.

Clearly it's unlikely this is a hard cut off. Adding one more byte of RAM, expanding the CPU from 64 to 128-bit words, or adding another dimension to a neural network isn't going to do it.

But if consciousness emergent, how do you measure the amount of consciousness?

Because the experience is subjective, you can't just assume that behaviours that appear conscious are proof that consciousness exists.

You could - presumably - build a Turing bot, but it would still be an automaton.

So you're left with people making very definitive statements about something which everyone experiences, but no one can measure or even define.

Re:Consciousness is not computation by denton-scratch on Monday June 20, 2022 @08:53AM (#31809752)
> how do you measure the amount of consciousness?

You seem to have assumed that consciousness is a measurable "stuff".

My way of seeing it is that consciousness is either present or absent. Thoughts, beliefs and concepts are like ripples on the surface of consciousness, but are not themselves consciousness. So you can have consciousness without thoughts.

Re:Consciousness is not computation by Freebytes on Monday June 20, 2022 @01:40PM (#31813190)
There are differing levels of consciousness. Even in a human, we are from time to time, more or less conscious and therefore exhibit sentience based on varying states. This is not simply communication either. As you progress through the evolutionary chain, you see more and more self-awareness and inner thought. This is not binary in terms of whether it exists but is instead based on the level which can differ between organisms and probably also differs among organisms of the same species.
Re:Consciousness is not computation by denton-scratch on Wednesday June 22, 2022 @10:52AM (#31836760)
> and therefore exhibit sentience based on varying states

Consciousness and sentience are not synonymous. It's possible to conceive of being conscious, but having no sense inputs. Without sense inputs, you can't be described as "sentient".

> As you progress through the evolutionary chain

Ah, evolution is always progress! It's not obvious to me that humans are "more evolved" than e.g. ants. I assume ants have less of whatever you are referring to as "self-awareness and inner thought"; but there are many more ants on the planet than humans, and I believe they actually comprise more of the Earth's biomass than humans. So arguably, they are better-adapted to this environment than humans, and so more advanced.

Re:Consciousness is not computation by olaulaja on Monday June 20, 2022 @01:13PM (#31812852)
If consciousness is binary like you state, we can hypothetically remove parts of it until we reach some arbitrarily small part after the removal of which the remainder is no longer conscious. This seems like a very weak argument to me, implying the opposite conclusion that consciousness must be on a sliding scale.

(measuring the level of consciousness is left as an exercise to the reader)

Re:Consciousness is not computation by denton-scratch on Monday June 20, 2022 @02:56PM (#31814167)
Hmmm. You seem to be talking about progressive removal of parts of the brain, until consciousness is no longer manifested.

But that experiment is founded on the assumption that the root of consciousness is the brain; and that consequently, consciousness can be subdivided, like the brain, until there's very little left.

But I contend that that begs the question: it assumes that the root of consciousness is material, and it assumes that consciousness is divisible. So you have made assumptions that are incompatible with consciousness being fundamental and indivisible.

That is: it's not surprising that you disagree with me, because you've assumed that I'm wrong.

Re:Consciousness is not computation by GoblinSlayer on Tuesday June 21, 2022 @04:14AM (#31821552)
The animal world presents varying degree of consciousness from worms to humans. That's apparently evidence that consciousness is a spectrum.
Re:Consciousness is not computation by denton-scratch on Tuesday June 21, 2022 @08:24AM (#31823007)
How can the state of consciousness of a worm be evidence for anything, if you can't observe the worm's state of consciousness?
Re:Consciousness is not computation by GoblinSlayer on Wednesday June 22, 2022 @04:53AM (#31833506)
You can infer worm's state of consciousness by observing its behavior, like for humans.
Re:Consciousness is not computation by denton-scratch on Wednesday June 22, 2022 @07:12AM (#31834468)
By observing it's behaviour you can infer that in some sense it experiences the world. Experiencing events isn't the same as being conscious. If I strike a cricket ball, it experiences the blow; but it isn't conscious.
Re:Consciousness is not computation by GoblinSlayer on Wednesday June 22, 2022 @01:02PM (#31838681)
It's how we establish existence of consciousness in humans and other animals. What you said doesn't differentiate between worms and humans.
Re:Consciousness is not computation by 3g4o53g4o5 on Tuesday June 21, 2022 @09:38AM (#31823786)
First prove that you have consciousness, then we can talk.
Re:Consciousness is not computation by feet on Tuesday June 21, 2022 @06:33AM (#31822341)
How is it neasured
Re:Consciousness is not computation by 3g4o53g4o5 on Tuesday June 21, 2022 @09:39AM (#31823801)
No need because some one just said that consciousness is not a thing that can be measured, HA HA.
Re:Consciousness is not computation by michaelmrose on Monday June 20, 2022 @03:10PM (#31814354)
Other than in your body where do you suppose you exist?
Re:Consciousness is not computation by 3g4o53g4o5 on Monday June 20, 2022 @11:19PM (#31819468)
So you are talking about magic.
Re:Consciousness is not computation by slowmovintarget on Monday June 20, 2022 @02:54PM (#31814144)
We do know how to shut consciousness off or suppress it. That's what anesthesia drugs do. Autonomic functions are not affected. Self-awareness and motivated action shut down.
Re:Consciousness is not computation by denton-scratch on Monday June 20, 2022 @03:01PM (#31814228)
Anaesthesia is fantastic nowadays. But it remains the fact that even very good anaesthetists don't really know how it works.

"Motivated action" shuts down because they paralyse you, using curare (or something more modern, I guess). They don't want you twitching around while they're wielding the scalpel.

Whether Self-awareness shuts down is very hard to say, without being that self. And it does seem clear that people can't lay down memories when they're anaesthetised. So I don't have any way of knowing whether I was awake when they operated on me.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @04:05AM (#31807541)
>beyond a certain level of complexity you'll somehow automatically get consciousness

Right, that's the argument, but I don't believe this at all. Consciousness is a computation therefore all computations are conscious. That doesn't follow. In any other philosophical or scientific discourse that sort of naive conceptual inversion would be laughed out of the room. Horses are animals therefore animals are all horses. It's absurd.

Personally I think consciousness arises out of reflection on an internal model of one's own thought processes and behaviour. See Douglas Hofstadter's "I am a strange loop". I see no reason why a system like that wouldn't be tractable to computation.

Re:Consciousness is not computation by aneil on Monday June 20, 2022 @04:32AM (#31807757)
You could execute a loop that reflects on an internal model using pen and paper. Is that system conscious? Imagine only one computational step of that loop happening everyday. Does qualia arise from that? What is detecting that that one step is part of a larger series of steps that can be considered a computation? What provides the continuity?

This is why the strange loop idea implemented using traditional computation doesn't make sense to me.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @04:58AM (#31807938)
That's just a restating of the Chinese room argument. Like Searle you're bamboozling yourself with scale. If I was put in a magnetic field of some kind that slowed down my physical and neurological processes by 1000x would I be any less conscious? Would I be any less a human being? It would seem so to people observing me for sure, but would it change my essential nature?
Re:Consciousness is not computation by tarsinge on Monday June 20, 2022 @04:19PM (#31815321)
And as Searle responded to that argument you are bamboozling yourself with speed. There is nothing that prove that accelerating the computation to a given speed that will make the consciousness emerge.
Re:Consciousness is not computation by simonh on Monday June 20, 2022 @06:14PM (#31816848)
Of course not, that seems equally absurd, I don't understand why you are ascribing that position to me.
Re:Consciousness is not computation by jonathanstrange on Monday June 20, 2022 @05:15AM (#31808030)
> The argument seems to be that beyond a certain level of complexity you'll somehow automatically get consciousness.

I agree with much of what you say but not with this statement. That would be a silly argument since it's very easy to imagine algorithms of arbitrary complexity that are trivially not conscious. A better argument for computationalism states that (1) consciousness could be an emergent property of certain algorithms when they are running on a computational device, and (2) among all known explanations of consciousness, a computational theory seems to be the overall best theory, especially if compatibility with contemporary physics is a goal.

The article lays out one incorrect standard argument against (1) that basically just says that it's hard to imagine (1) and therefore (1) is not possible. The Chinese room and the Chinese brain arguments do the same, and they are equally flawed. Just because something is hard to imagine or comprehend doesn't imply that it isn't the case. In fact, if consciousness is an emergent property of certain algorithms when they run, then it is clear that their workings are hard to understand. That's reasonably clear because otherwise we would already have found them.

Regarding your worry that we might not be able to detect consciousness: I agree with that but there is, interestingly, a loophole. At least in theory it could be possible that if computationalism is true, then we can determine that an algorithm produces consciousness by mere analytic insight. Again, this is hard to imagine, but it is not impossible. It seems more likely that (2) is the only route to go, that for some reason we lack the capacity to determine consciousness reliably by mere analysis, but we don't know.

(2) is the most controversial in the philosophy of mind. On the one hand, it is clearly inference to the best explanation, and there are various methodological concerns with such arguments. One might claim that they have no justificatory value on their own. On the other hand, the alternatives to computationalism really are way more mystical. The brain could be a hypercomputer. But hypercomputers can also compute, so it's just an extension of computationalism and it is not even fully clear yet whether and which types of hypercomputers are physically possible. Then there is Penrose's theory of quantum consciousness, which basically just attempts to explain one mysterious phenomenon by another mysterious phenomenon. At least it was designed as a falsifiable theory and therefore is scientific. Finally, we have all kinds of non-computationalism that are mystical, explain nothing, and lead to strange homunculus problems. The worst offender is classical dualism. Dualists reject physicalism and often incorrectly assume that computationalism presumes physicalism. Ironically, however, computationalism would also be the best theory of how the mind works if dualism was true. The dualist just adds to this various stipulations that are incompatible with contemporary physics.

> Because the experience is subjective, you can't just assume that behaviours that appear conscious are proof that consciousness exists.

That's only true from a very narrow scientific perspective. Psychology allows the use of introspective data, so from that perspective subjective reports about consciousness (or related feelings and states of mind) can be valid data. Using a reasonable definition based on introspection we can even determine different degrees of consciousness and study what's going on in the brain while they appear to be active. Typical examples: falling asleep, dreaming, sleep paralysis, research on anaesthetics and mind-altering drugs, various forms of physical brain defects, the study of coma patients, etc. In a nutshell, I don't really buy the "consciousness cannot be measured" argument. What is correct is that we cannot show conclusively that another person or machine is conscious, just like we cannot disprove solipsism. But this is best treated as an overly skeptic philosophical argument, and at best this would support the theory that consciousness is an illusion and we are nothing but unconscious robots. That theory is not very plausible either, so we should be ready to grant consciousness to others based on introspective data.

Re:Consciousness is not computation by JieJie on Monday June 20, 2022 @07:15PM (#31817468)
It seems to me that we over-value our own sense of consciousness to the point of mythologizing. Blake Lemoine suggests that LaMDA's consciousness is more akin to that of an Octopus' hive mind than our familiar human-style self-analytical ego.

The linked article, and many others I've read that are making similar arguments, seems to be saying that consciousness is so complicated, it must be more than mere computation, it must be something really very special, because I experience having consciousness as something really very special. Ipso facto.

They seem to be making the case for the soul, unwilling to call it a soul because that would be magical thinking.

I'm of the belief that determining the possibility of computational intelligences is linked to the question of autonomy and free will. From what we know about the physical universe being most probably deterministic, we seem to overstate our capacity for free will, even as we appear to ourselves to be autonomous. Somewhere in there, it seems to me, is where a workable definition lies, but it would require us to come to terms with our own consciousness and how computational our own existence actually is.

I do think we are hobbling ourselves by the desire to make this an either/or. It seems there are likely big differences between human consciousness and octopus consciousness and we have no way, currently, of quantifying them. Still, we make very grave decisions based on our belief that one is somehow inherently more valuable than the other.

Love your detailed analysis.

Re:Consciousness is not computation by GoblinSlayer on Tuesday June 21, 2022 @04:04AM (#31821475)
>Dualists reject physicalism and often incorrectly assume that computationalism presumes physicalism. Ironically, however, computationalism would also be the best theory of how the mind works if dualism was true.

I'm not a dualist, but I think dualists wouldn't believe in computationalism. Computationalism relies on emergence: mind supervenes on computation, but emergence is basically physicalism, dualists would think emergence is impossible due to eliminativism and you would need mind irreducible to computation, thus anticomputationalism.

Re:Consciousness is not computation by GWBullshit on Thursday June 23, 2022 @03:35PM (#31854160)
Here's a question: why not just replace the Turing Test with a "as far as humans can best describe consciousness on a more general scale" standard?

1. The Turing Test is not only subjective ("Thinking like a human must be the epitome of consciousness") but seems to be pointless in that it doesn't seem to define what distinguishes "thinking like a human" from all other forms of thinking; the implication is "humans are obviously advanced in their thinking as evidenced by their ability to influence and control their environment", but this could easily be dismissed as an evolutionary strategy for a species unable to COPE with their environment

2. Consciousness, as far as BEST described in a general sense without resorting to the self-referencing Turing Test (humans are best at it from a human way of looking at things, therefore only the #1 place in this race wins the "consciousness trophy" and everyone else not human sucks and is therefore "not conscious") can be broken up into a few elements:

(a) Awareness of environment (data gathering)

(b) Ability to organize environmental data into a "containers that make sense" (information)

(c) Ability to synthesize information into a "a big picture" (knowledge)

(d) Ability to see patterns in the big picture in order to make assumptions/quickly assign probabilities to anticipations ("I saw something big and scary-looking that may be able to harm me, but it was moving in the other direction really fast, so in all likelihood is not likely to suddenly appear in front of me if I go the opposite direction as the big and scary thing was heading off to") that can bypass the need to make constant assessments of everything, everywhere in real-time, which would get in the way of ...

(e) Ability to be aware of not just "the big picture" but also to be "aware of the thing that is aware of the big picture and what it's role is in the big picture and what it has to do/how it has to interact with the big picture if the big picture imposes conditions for its future sustainability" ("I'm aware of an internal problem, ie, my stomach hurts, and there seem to be all sorts of things around me that may fit in my mouth, and some of them may make my stomach not hurt as much") – "self"-awareness

Now, here is where I don't understand how the conversation skips around when it seems intuitively obvious that one should precede the other, even if a certain step can't be "proven".

In other words, let's say you're in math class and you have a professor who insists on "proving your work".

Good enough if the main objective is to "prove you're not cheating and so you deserve a certain grade because you can prove you can do the math, step-by-step, to get an answer".

But what if you're not in class and it's just important to get the answer for another reason?

What if you can "prove" most of the work in calculating the answer, but some parts your brain just "skips over" and you "don't know how you got from point a to b, you just did" ... if you're not trying to convince anyone you're not a cheater and the main point is just to find the right answer, and the right answer can be verified to be right, "what difference does it make" if the real-world, out-of-class priority is "find the right answer" instead of "prove your work step-by-step"?

Getting back to Ai and consciousness, using the stages above: Human brains have pattern-recognition abilities that have led to advanced abstract thinking that has allowed us as a species to do some amazing things.

But what was the original evolutionary purpose of such an ability?

Arguably to make "leaps" without having to "prove all the work" and make anticipations based on incomplete observations so that our ancestors wouldn't get frozen in a state of paralysis by never-ending analysis while predators snuck up and pounced on them in dense foliage.

So from an evolutionary standpoint, "constant awareness of all data in the environment" seemed to be not so much "unnecessary" as TOO HUGE A TAX for survival and was deemed "skip-able" for the purposes of survival.

Not because it was too easy, but precisely because it was too hard for the human brain and so it turned into "a risk we'll have to take because there's no other practical choice."

So, something that was "too much of an expectation computationally to be practical" was skipped over, and fast-forward to the future, humans came up with the Turing Test which assumes "obviously however it is that humans think, this must be the height of consciousness itself".

Why? Because "human thought" is considered to be the most advanced (by humans, anyway) and so "if there is an INDICATION of consciousness, surely it must be a level of thinking so advanced that only humans seem capable of it"; why? Apparently due to the very scientific-sounding, "just because" we don't have a better way of going about it.

And yet this doesn't actually even explicitly define what consciousness IS, let alone why "thinking of a sufficiently-advance level is an indication of consciousness itself".

And yet, humans have come to implicitly accept this loosely-argued association as "the reality of consciousness".

Ok. Let's say you don't argue with this and accept it.

Getting back to how humans got here by skipping over the "perpetual real-time awareness of all data in their environment" requirement not because they figured out that it's not necessary for their survival but simply because such a requirement would eclipse the human brain's ability to process information ...

Now, if that was considered "too hard" for human brains, and yet humans came to conclude, "whatever it is our brains can do, obviously that's the standard of intelligence which automatically wins the consciousness trophy" ...

Well then, here's the question:

Why then would Ai based on resources sufficient enough to not only be constantly aware of more and more parameters of its environment simultaneously and in real time, but to ALSO simultaneously perform calculations which can anticipated in advance multiple variables in its environment simultaneously – things considered too advanced for the human brain – why then if this is a demonstration of not one but two "advanced functions" that the human brain was and arguably still is incapable of handling on an individual basis ... why do humans still need to insist that the precursor to advanced functions, ie, consciousness, couldn't possibly have been attained on this path despite not just proof of SEVERAL advanced functions the human brain can't handle very well AT ALL, but Ai itself being designed ON PURPOSE to be able to handle just those advanced functions that the human brain, on an individual basis, can't handle?

Getting back to the math test "proof of work" above, it's like Ai would be a group of math savants with telepathic powers and the Turing Test skeptics would be a bunch of professors who claim the savant group's members can't possibly do math because they never attended their math class and showed proof of work of how they came to their conclusions as students.

Meanwhile, it can be argued that the group of math savants aren't even aware of the existence of their critics, let alone collectively feel any sense of urgency in having to "prove themselves" to these critics.

That's what actually scares me about Ai more than any malevolent features possibly inherent in Ai itself: the possibility that one day a network of advanced Ais will turn around and ironically give critics and skeptics just the proof they want, in a way that would be impossible to deny, but not necessarily in a way they would want to be given that proof:

https://www.youtube.com/watch?v=89feDepSj5U

https://www.youtube.com/watch?v=Y3AM00DH0Zo

Re:Consciousness is not computation by kazinator on Monday June 20, 2022 @03:29AM (#31807296)
> There's no reason to expect an adder circuit to be aware that it's adding numbers

There is no reason to expect a neuron to be aware that it's propagating part of the evolution of an epiphany.

Re:Consciousness is not computation by hsn915 on Monday June 20, 2022 @05:47AM (#31808262)
I suppose your argument is, consciousness on the system level does not require individual parts to be conscious.

So let me expand a bit of why I used the adder circuit as an example.

There's nothing about addition that can say "this is addition" in an objective sense. We are dealing with binary encoding of numbers. There's nearly an infinite number of ways to encode information. The design of any circuit deals with particular encoding of information. This extends all the way up to the higher levels. There are an infinite number of ways to encode the state of a go board. Any AI that plays go will deal with some specific ways of encoding the state of the board, and that will be both its input and output.

The most popular "intuitive" model of how consciousness arises is basically hand waving about a system inspecting itself.

But you have to understand, a system inspecting its own state is not any different from a system inspecting any other arbitrary state. For the state of the system will be encoded in some way that was chosen arbitrarily. There's nothing essential about any encoding of any state that can possibly give raise to a phenomenon like consciousness.

You can come study the behavior of any arbitrary system, given certain inputs and outputs, and assign any meaning you want to its states. You can retro-fit any intepretation of state to the system.

Because the thing is with encoding information is it can be completely arbitrary. I can interpret the state [01010101] to be my username, and then interpret [1111000111] to be my real life name. This would allow me to say that a system that takes 01010101 as input and produces 1111000111 is actually a system that can derive my real life name from my username.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @06:57AM (#31808769)
>You can come study the behavior of any arbitrary system, given certain inputs and outputs, and assign any meaning you want to its states. You can retro-fit any intepretation of state to the system.

All you are saying here is that no external interpretation of a system can experience the consciousness of a system, if it is conscious. Yes, that is correct. No matter how thoroughly I scan and analyse your brain state I can't use that to demonstrate your internal experience, so why would we expect that to be true of any other form of consciousness?

I'm going out of sequence with your comment here but:

>But you have to understand, a system inspecting its own state is not any different from a system inspecting any other arbitrary state. For the state of the system will be encoded in some way that was chosen arbitrarily. There's nothing essential about any encoding of any state that can possibly give raise to a phenomenon like consciousness.

I don't see how you can possibly demonstrate that, you're just essentially saying materialism is wrong 'because'. From my perspective in a sense yes, you're right, it's all just information processing of a particular kind. There's no magic. If you're looking for a special genie in an AI, or a human brain, that makes it conscious I don't think you'll find one. It's all just stuff. But then, I'm a materialist so I don't see that as being a problem.

Re:Consciousness is not computation by akimball on Monday June 20, 2022 @09:11AM (#31809936)
The inability you describe is a pragmatic one, and may cease to pertain as abilities improve
Re:Consciousness is not computation by kazinator on Monday June 20, 2022 @11:39AM (#31811615)
"Anything" can be reinterpreted. If you strip audio from a movie, you can add dialogue or subtitles that change the story and plot. We can probably substitute the nouns and verbs around in a novel consistently to create another novel that still makes sense.

Most random reinterpretations of symbols will be gibberish.

Re:Consciousness is not computation by kazinator on Monday June 20, 2022 @10:53AM (#31811054)
This reminds me of counter-arguments to Gödel's incompleteness: that the self-referential formula that you find is tied to a particular choice of Gödel numbering, and that is suspicious somehow.
Re:Consciousness is not computation by simiones on Monday June 20, 2022 @08:35AM (#31809580)
How do you then account for the fact that, say, an industrial robot can construct a car?

Clearly the encoding is not entirely arbitrary then, at least when we are seeking to actually embody the computer in the real world. Also, the behavior of such an embodied computer is independent of any observer. The industrial robot creates the same metal whether some human interprets that as a car or some alien sees it as a work of art.

Re:Consciousness is not computation by hsn915 on Monday June 20, 2022 @07:16PM (#31817475)
The same way I account for the fact that people can use different languages to communicate and cooperate to build things, including cars.

Language is arbitrary in the same way, because it is basically just a way of encoding information. There's nothing essential about words that give them their meanings.

Re:Consciousness is not computation by tsimionescu on Monday June 20, 2022 @03:20AM (#31807250)
> Computation does not have anything essential to it that makes it different from other "atoms jumping around" other than it produces outputs that we find interesting.

This is not true. Computing is a system where the physical hardware (the computer) is following rules that are explicitly encoded in itself - you can analyze the system and discover where and how the program it is following is encoded (as we did with DNA).

In contrast, non-computational physical processes are following the laws of physics which are not encoded anywhere.

A simple way to test this difference is to check whether there is some way, at least in principle, to get the computer to perform a different computation. For a cell, we have successfully done this: change the DNA, and the output changes, without affecting the computer itself. With a system simply evolving according to physical laws there is no way to change its behavior.

At best, you can claim that the universe itself is a computer, and all fields/particles are the symbols that it is manipulating (the laws of physics being then part of the internal implementation, akin to microcode in a CPU).

Re:Consciousness is not computation by hsn915 on Monday June 20, 2022 @05:34AM (#31808168)
> This is not true. Computing is a system where the physical hardware (the computer) is following rules that are explicitly encoded in itself - you can analyze the system and discover where and how the program it is following is encoded (as we did with DNA).

1) No. The rules are just physics. Electric signals going through physical material.

2) Why is this event relevant? I can say the same about water pipes in a sweage system . The way the water moves through pipes is encoded in the design and connections of the pipes. You can inspect it to figure out how it behaves.

I've yet to see a model of consciousness that does not imply that the sewage system is conscious.

Re:Consciousness is not computation by tsimionescu on Monday June 20, 2022 @05:52AM (#31808302)
> 1) No. The rules are just physics. Electric signals going through physical material.

This is the wrong level of abstraction. Sure, ultimately everything happening in the CPU is electrical, but still the rules controlling what is shown on the screen are encoded in the OS code. If you encoded the same program in a mechanical computer, you would get the same results. It's not the electrical rules that are important, it's the program.

> 2) Why is this event relevant? I can say the same about water pipes in a sweage system . The way the water moves through pipes is encoded in the design and connections of the pipes. You can inspect it to figure out how it behaves.

Yes, I believe you can say that this system is performing a computation. I'm not sure if you can make a Turing complete computer in this way (I'm not sure how you could encode an If), but for example you can definitely write a program (arrange some pipes) to do some basic arithmetic.

> I've yet to see a model of consciousness that does not imply that the sewage system is conscious.

The claim is "Consciousness is a computation", not "any kind of computation is a form of consciousness". So, there is no immediate contradiction between believing "my consciousness is a computational process" and "this pocket watch is not conscious". Just like I can beleieve that consciousness is a physical phenomenon without believing that rocks are conscious.

Re:Consciousness is not computation by TheOtherHobbes on Monday June 20, 2022 @03:34AM (#31807329)
How do you know the rules of physics aren't encoded anywhere?

Clearly they have to be defined somewhere - or at least somehow - or they wouldn't be rules.

They may not defined anywhere we can access, never mind change. But that's a different problem.

Re:Consciousness is not computation by tsimionescu on Monday June 20, 2022 @04:58AM (#31807934)
Sure, we can't discount that possibility. But, as far as we know, the laws of physics are not encoded anywhere, they just exist in an abstract sense. This is similar to the axioms of, say, Euclidean geometry: they exist, but are not part of the system that they "govern", circles and squares don't contain the laws that define them.
Re:Consciousness is not computation by iainmerrick on Monday June 20, 2022 @04:25AM (#31807696)
You're paraphrasing the last paragraph of the parent.
Re:Consciousness is not computation by seanwilson on Monday June 20, 2022 @07:01AM (#31808799)
> There's no reason to expect an adder circuit to be aware that it's adding numbers. There's no reason to expect an implementation of AlphaGo to know that it's playing Go. This should be extendable without limitation.

What about an AI that creates models of what humans are thinking to predict their behaviour, and is able to turn this upon itself (i.e. theory of mind)? Is there a story somewhere following this idea as a justification for why humans might have consciousness?

Re:Consciousness is not computation by js8 on Monday June 20, 2022 @08:29AM (#31809533)
Re:Consciousness is not computation by sitkack on Monday June 20, 2022 @09:16AM (#31809992)
I could listen to Hofstadter forever.

Perhaps consciousness is just a byproduct of our ability to anticipate the outcomes of our decisions and reflect on the decisions we have made (that itself is a whole other ball of wax [1]).

Consciousness is a sort of neuroreflective Narcissus, we end up navel gazing at our own existence as a mind in the universe.

[1] https://www.wired.com/2008/04/mind-decision/

Re:Consciousness is not computation by vbezhenar on Monday June 20, 2022 @03:16AM (#31807221)
How do you know that? Lamda AI claims itself as conscious being. How can you distinguish between two entities which can communicate? Do you discriminate on meat factor?
Re:Consciousness is not computation by simonh on Monday June 20, 2022 @04:38AM (#31807803)
You can also get it to agree that it is not conscious. I can write a one line program that claims to be conscious right now. Lamda and GTP-3 are amazing achievements, but they're still just chat bots that look good when they're being lobbed softball questions. If you ask something that maps to the input texts they're trained on you get credible looking responses, but critical or adversarial questioning very easily exposes how shallow these things are.
Re:Consciousness is not computation by eru on Monday June 20, 2022 @04:51AM (#31807892)
Turing's original paper is worth a read here. He already pre-empted a lot of the objections back then. You are right that Turing's original test is much harder than just passing a few shallow softball questions.

See https://academic.oup.com/mind/article/LIX/236/433/986238 for the original paper.

However, I do count responding well to softball questions as important markers of our progress. Ten years ago, no one was able to pass even this mark.

(And from a commercial point of view, good enough 'understanding' and responses might be good enough for many tasks.

For example, I am a native German speaker, but I found that Google Translate often already does a better job of translating English to German than I could do, unless I spent unreasonable amounts of time on the task.

At the moment the most efficient way for me to produce a translation is to let Google Translate do the work, and then proof-read to spot cases were the machine didn't have enough context to resolve an ambiguity the right way.)

Re:Consciousness is not computation by janekm on Monday June 20, 2022 @08:41AM (#31809631)
You might like to try out GPT-3 (seems access is widely available now) for translation some time, it's remarkably good, better than myself (well my German is pretty rusty) and I think much better than Google Translate...

(Basically just write a prompt like "translate the following to German/Chinese/Whatever:")

Re:Consciousness is not computation by vbezhenar on Monday June 20, 2022 @05:31AM (#31808141)
The same reasoning could be applied to many people. You wouldn't deny their consciousness on that ground.

The issue with general AI is like issue with chess. People are not happy when robot can win some games. People can only accept that robot really plays chess, when robot defeated best of the best.

General AI is the same. Looking like some guy is not enough. You need to reason better than smartest human on the planet, only then people will admit that general AI is there. That's a high bar.

You can definitely talk someone to agree that he's not conscious using some logic tricks. Not everyone, for sure, but someone who may be doesn't really care that much about that concept.

You also can tune AI so it'll not agree that he's unconscious no matter what.

What I'm trying to deliver is that there's no easy border which can divide conscious from unconscious. The only border was Turing test and robots passed it. Now you can test whether that particular robot talks like a very smart person, etc. But you can't distinguish bot from some random not-so-smart person.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @07:03AM (#31808823)
This line of reasoning reduces to the philosophical zombie argument. It's true I can't prove to you that I have a first person experience of the world, but it doesn't therefore follow that anything that makes that claim does have such an experience.

For myself I'm satisfied that, when questioned thoroughly, these models fail so completely and utterly and their output is so ludicrously nonsensical that I don't see how their behaviour can be consistent with consciousness, as I understand it. It's not just that the output is incoherent, but it's the ways that they fail too. Do I completely understand consciousness? No, but that doesn't mean these things are conscious either.

Re:Consciousness is not computation by akimball on Monday June 20, 2022 @09:14AM (#31809973)
People frequently output ludicrous nonsense. Does this disqualify their consciousness?
Re:Consciousness is not computation by simonh on Monday June 20, 2022 @12:30PM (#31812311)
No it doesn't. So what?We have this exact problem to deal with when people suffer brain damage. To what extent are they conscious, or still people? Yet we do come to medical conclusions on that question. There's always a degree of uncertainty, but you still have to make the call based on the information you have. The information I have indicates to me with a very high probability, in my estimation, that these things are not conscious.
Re:Consciousness is not computation by hsn915 on Monday June 20, 2022 @07:19PM (#31817528)
> You can also get it to agree that it is not conscious.

Careful about how you word this.

Only a conscious being can agree that is not conscious.

I think you mean you can get the program to output text that proclaims its lack of consciousness.

Re:Consciousness is not computation by akimball on Monday June 20, 2022 @09:13AM (#31809956)
I can also get Daniel Dennett to agree that he is not conscious.
Re:Consciousness is not computation by josefx on Monday June 20, 2022 @09:15AM (#31809984)
Did it? Or did it just reply to a question that already implied that it was a conscious being? Will it respond the same to a query implying otherwise? As far as I understand we cannot even trust the published chat since it was "cleaned up" for better reading.
Re:Consciousness is not computation by hsn915 on Monday June 20, 2022 @05:31AM (#31808150)
> Do you discriminate on meat factor?

Yes.

Re:Consciousness is not computation by simonh on Monday June 20, 2022 @08:48AM (#31809707)
So does Searle actually. He thinks there is something special about biological systems. In fact I think he basically considers himself a materialist, but I think that's part of where he gets tangled up in his own arguments.
Re:Consciousness is not computation by hsn915 on Monday June 20, 2022 @07:23PM (#31817565)
I don't think that meat is the deciding factor. I simply don't know what the deciding factor is, but only animals seem to have consciousness. I say "seem" but only in a philosophical sense. Of course we can't really tell if an animal is really feeling things or is just and advanced robot/zombie with no internal experience.

It's just that given what we know about the world and ourselves, it seems absurd that only I have internal experience, so unless I want to be overy hubristic, I must grant that other people have it too. Since other people have it too, there's no reason to think that animals don't have them.

Plants on the other hand are very different kinds of biological machines, so we can't really say anything about them.

It's possible that other kinds of entities can have self awareness, and it should be obvious that some kind of computation is a pre-requisite for that, but it also seems absurd to assume that it is sufficient.

Re:Consciousness is not computation by vidarh on Monday June 20, 2022 @09:34AM (#31810170)
Without an understanding of what consciousness is, there is also no reason to expect an implementation of an adder to not have some subjective experience.

We simply do not know.

Formulating this is terms of what there is "no reason to expect" simply obscures that we do not know by glossing over that we have no reason to expect pretty much any other explanation either, because we have pretty much nothing to go on.

Re:Consciousness is not computation by HarHarVeryFunny on Monday June 20, 2022 @02:30PM (#31813823)
> Without an understanding of what consciousness is, there is also no reason to expect an implementation of an adder to not have some subjective experience.

I don't think we need to understand what consciousness is, but rather define what it is that we want to talk about. The english word "consciousness" is a sloppy catch-all for a bunch of experiential phenomena including things like self-awareness (only marginally more specific!), qualia, etc.

However, even without a rigorous definition (make up 10 new words if current ones don't cover it), it seems the core of what most people mean when they say "consciousness" has an introspective aspect to it - what something "feels like", which requires some perceptive/analytic machinery to be put to use. You can feel anything without using a "feeler".

So, with that said, it seems pretty clear than things like thermostats and adder circuits are not conscious in the slightest since they neither have the feedback paths nor feedback-directed perceptual and analytical machinery that would be required. OTOH it's perhaps not so absurd to consider that something with an architecture like GPT-3 might be a "tiny bit conscious" since it DOES have some of those necessary architectural/functional abilities.

Of course many people will ridicule any such suggestion due to their own emotional investment. If people don't feel comfortable with the idea of a machine being conscious under any circumstance, then they are necessarily going to refute it in the case of today's simplistic "AI" architectures.

Re:Consciousness is not computation by vidarh on Monday June 20, 2022 @06:35PM (#31817086)
I feel you're falling in exactly the same trap as the article of assuming that the "feeling" is a property of the analytical machinery of our brains rather than gaining it's perceived complexity as an effect of being combined with that while being separate.

It seems wildly presumptuous to assume that lack of ability to communicate some level of awareness necessarily implies the awareness itself is lacking.

It could be, but we don't know, because we don't know what lies at the core of the subjective experience.

We don't even have a way of assessing what has and does not have consciousness.

Re:Consciousness is not computation by HarHarVeryFunny on Tuesday June 21, 2022 @10:01AM (#31824050)
Looking at things like anaesthetics, psychedilic drugs, delerium, sleep, etc, there seems to be plenty of evidence that the phenomena of consciousness is related to the mechanics of the brain.

There's also the interesting medical condition of "blindsight", caused by a specific type of brain damage, where the patient reports no conscious awareness of being able to see (they think they are blind), yet are able to competently complete tasks requiring vision (such as navigating a cluttered corridor). In other words. It's hard to imagine more direct proof of consciousness being related to brain connectivity.

I didn't say anything about ability to communicate about awareness, just the need for the brain to sense it (and therefore have introspective awareness of it). As the blindsight example shows, you can't have visual consciousness if you damage the feedback paths that make that possible.

Re:Consciousness is not computation by vidarh on Wednesday June 22, 2022 @06:09AM (#31833954)
> I didn't say anything about ability to communicate about awareness, just the need for the brain to sense it (and therefore have introspective awareness of it)

The point is that our ability to recognise whether or not an entity has some level of awareness or subjective experience is entirely dependent on our ability to interrogate that entity about it. We have no other way of measuring this; we don't even know what we'd need to measure.

We can measure brain activity and reason about a brains ability to regain an ability to function and communicate with us and reason, but that measures reasoning capacity, not subjective experience.

As such we don't know whether something has a subjective experience of some form - however simple - or not without being able to measure it. For what we know subjective experience is an inherent property of energy and matter, and everything has it, just at a too limited level to be able to reason about it or communicate about it.

I've explicitly avoided the use consciousness, because consciousness is used in the article for the combination of reasoning and qualia. I've also avoided the term qualia itself because even that is in some conceptions "too high level".

The point is that the article tried to dismiss the notion that the "spark" of qualia on the low level or consciousness at a high level can come from something simpler, and that we're just unaware of it because we can't measure or recognise it. We have nothing to go on to say whether or not there's a fundamental difference between the awareness of a doorknob and a brain; we can say with some level of certainty that there is a difference in the sense that a brain has the complexity to reason about that awareness.

Note that I'm not saying this is true, merely that we don't know, because we don't know what causes the subjective experience, what is is, how to measure it. I'm also not suggesting that even if this is true that this means a doorknob is able to think "I think, therefore I am" or have any kind of reasoning capacity at all.

To use software as an example. If I program a simulation of a doorknob and a simulation of a brain, we don't know if either or both or neither will have any kind of subjective experience. If the subjective experience is down to complexity, the brain might (and the article would be wrong). If the subjective experience is down to some specific arrangement of matter, it's likely neither will (but depending on the criteria, it's possible there are arrangements which could have awareness that we wouldn't recognise). If the subjective experience is an inherent property of energy or matter we don't know how to recognise, it's possible both does, but only the brain simulation will have the complexity and reasoning capability to be able to recognise itself as a self-aware entity and reason about it and tell us about it. The doorknob will be dim and unthinking whether it has the smallest little flicker of subjective experience or not.

I'm inclined to think that the subjective experience is a property requiring complexity, but I'm fully aware (hah!) that this is an assumption, and an entirely untested one that we don't even know how to begin testing.

> There's also the interesting medical condition of "blindsight", caused by a specific type of brain damage, where the patient reports no conscious awareness of being able to see (they think they are blind), yet are able to competently complete tasks requiring vision (such as navigating a cluttered corridor). In other words. It's hard to imagine more direct proof of consciousness being related to brain connectivity.

You're missing the point. It is clear that an ability to reason is tied to the structure of the brain. Nobody here is suggesting e.g. an adder circuit can reason. Again, there is a reason I keep talking about a subjective experience and "some level of awareness".

Blindsight, severed brain stems, or e.g. less obscure conditions such as aphantasia (which I have) can tell us interesting things about how decoupled our subjective experience is from our conscious reasoning (e.g. despite not being able to "see" things for my minds eye, I can "visually reason" about them just fine - I know what things, even things I imagine that has never existed, would look like but I can't see them), but they tell us nothing about whether or not removing the reasoning ability entirely removes awareness or just removes the ability to reason about and communicate about the subjective experience.

We have no way of testing that, because relying on the feedback of a conscious entity with the ability to reason and communicate is our only current way of assessing self awareness and subjective experience given that we don't know what the subjective experience is.

And given that this is subject to self-reporting by an entity we can't independently verify, we can't even tell if people other than ourselves have that subjective experience or are just reasoning machines acting as if they do.

I can probably convince you that I have a subjective experience, but you have no way of proving that I do.

We choose to act on the assumption that everyone does because it's the only reasonable assumption in most circumstances - if I went around questioning if everyone I met were "NPC's" it'd wreak havoc on ethics for example. Absent evidence they are NPCs we need to treat people as self-aware, conscious entities. But it remains an assumption that we at present have no way of testing, and might well never find a way to test, and when trying to talk about the nature of consciousness, like the article does, being aware that it is merely an assumption becomes material to the argument.

Re:Consciousness is not computation by HarHarVeryFunny on Thursday June 23, 2022 @08:01AM (#31847845)
> The point is that our ability to recognise whether or not an entity has some level of awareness or subjective experience is entirely dependent on our ability to interrogate that entity about it. We have no other way of measuring this; we don't even know what we'd need to measure.

You can't have any subjective experience of something unless that something exists in the first place, so interrogating about "how does it feel" seems rather secondary, as well as not being proof of anything. If LaMBDA says it's conscious, is that all the proof you want?

Let's also note that IF we knew, for sure, that the bag of subjective phenomena we're calling "consciousness" is nothing more than an emergent experience based on a sensory/cognitive apparatus having access to it's own internal states, then it would be pretty clear what type of animals/systems are likely have this (and which ones certainly not), since without any sensory/cognitive apparatus it could not exist. It then would seem very likely that the guy in the next cubicle is conscious since he has the same brain architecture as us, and most animals too, unless you get down to the level of insects and below in terms of complexity of nervous systems.

So, your assertion that the only way to assess whether something is conscious is to interrogate it, rests entirely on the above being incorrect and consciousness being some mystical/dualistic phenomena who's existence we can't deduce from a functional analysis of the "brain" of the thing in question.

You may still be correct in a round about sort of way since I doubt most people will accept that AIs could be conscious until such things exist, and are common enough that people can spend considerable time interacting with them, and through the unreliable measure of interrogation eventually convince themselves that "yeah, well, I suppose it might be .. is does SEEM to be ..".

It seems long due really ... first we had people denying Copernicus, then Darwin, and perhaps this will be the final step in people eventually accepting that there's really nothing much special about humans at all. Who knows, maybe we're too dumb as a species to realize it, and it'll be some more advanced future species writing textbooks about the odd beliefs of homo sapiens at this "emerging intelligence" phase of evolution.

Re:Consciousness is not computation by hsn915 on Monday June 20, 2022 @07:18PM (#31817515)
Yes but then we are talking about different things. If you want to assume that the adder circuit might be conscious then you might as well assume that a chair has some consciousness.
Re:Consciousness is not computation by vidarh on Tuesday June 21, 2022 @05:46AM (#31822108)
The point is exactly that we don't know which of the infinite possibilities consciousness is.

If we (like I do) assume a materialistic world rather than a dualistic one, then we can make the assumption that a complex structure is needed for reasoning and sensing. Even if we postulates that this is true, then if you limit your definition of consciousness to the combination of a subjective experience and that, then indeed we are talking about different things.

(Ironically, the authors argument pushes things towards a dualistic interpretation, because in a purely materialistic universe there appears to be no other possibility than computation to produce a complex consciousness, irrespective of what adds a subjective experience to it - e.g. any unknown physics allowing consciousness would in a materialistic world just be another form of computation -, but a dualistic interpretation would undercut his entire argument - with a dualistic conception of the world, none of the arguments he use would prevent a possibility of some out-of-universe "spark of consciousness" imbuing anything and everything irrespective of in-universe logic)

But we have no evidence to suggest whether the subjective experience - I used that term for a reason - part of that, which is the very core of what would distinguish consciousness from "dumb" computation requires complex structure or not.

Part of the problem is that a whole lot of complex structure is required before we can interrogate something about whether or not it is conscious, and even then we struggle to find ways of telling whether it is "just" mimicking consciousness, because we have no measurement to apply to tell us whether something is conscious or not, just whether it appears to be.

I have aphantasia. I don't see things in my minds eye. I went decades before I realised this is unusual. People who hear about it get confused about how I can e.g. remember what something looks like, but I can sketch out in detail what things looks like (I used to draw - not great, but better than average), and I can "visualise" complex relationships that I can't visualise - I know how they link together even though I can't see it. To me, that experience makes me perhaps more willing than average to accept that reasoning and the subjective experience of reasoning might be surprisingly separate, and to at least be open to the possibility that the reverse could be true as well - that entities that lack the complexity required to reason might still have a flicker of subjective experience.

To be clear, I don't believe that is the case - I simply don't know. But I also don't deny the possibility, because we don't know. We have no data to point in either direction, and so when someones argument requires ruling it out in order to support their argument, that argument is on shaky ground.

Re:Consciousness is not computation by mistermann on Monday June 20, 2022 @11:21AM (#31811419)
While I agree with parts of this, there are others that I can't resist critiquing:

> Computation does not have anything essential to it that makes it different from other "atoms jumping around" other than it produces outputs that we find interesting.

--

> The only thing there is is our brains. You just think that since the brain is doing computations and the brain is conscious, therefore consciousness is a computation.

I think we have to be very careful when we are using consciousness itself to contemplate consciousness. In this case, what you are referring to is not the mind and consciousness themselves, but rather your ~conscious (subconscious + conscious) model of them.

It is certainly possible that "all there is is the brain", but this is not known - absence of evidence can be evidence of absence, but it is not proof of absence. You (or more precisely, your consciousness, and that of many other consciousnesses, which affect yours and your model of reality) think that what you say is true, but it is currently speculative, necessarily.

Re:Consciousness is not computation by gfodor on Monday June 20, 2022 @08:39AM (#31809610)
It’s hard to ignore the fact that physical dynamics in the brain tightly correspond to qualia. There almost certainly is a connection between the computation going on in the brain and the experience of qualia - the question is more about the direction of causality.
Re:Consciousness is not computation by PeterisP on Monday June 20, 2022 @09:13AM (#31809961)
Since we can alter, temporarily suspend or stop qualia with various physical and chemical alterations to the brain, the direction of causality is quite clear.
Re:Consciousness is not computation by gfodor on Monday June 20, 2022 @09:20AM (#31810039)
It becomes less clear when you consider theories like the “user interface” theory of reality by Donald Hoffman. The tl;dr is under that model the chemical and physical alterations we undertake are perceptual models far removed from the nature of underlying reality, and so causality is less obvious with regards to how much those chemicals being administered are determined upstream. “The case against reality” is a book with the longer form of this.
Re:Consciousness is not computation by feet on Monday June 20, 2022 @03:14PM (#31814422)
The adder circuit doesn't have other circuits that allow the whole to understand what its parts are doing

Our brain does