If interconnected meat, tissue, water, and fat can be conscious, why not something else?

You think this is some special matter? It's just carbon atoms all the way, which even started out as simple chemicals in a pond.

And just like you can't see "consciousness" in the brain (it's just a piece of meat with some electrical signals moving inside), you can't see it in the rocks either.

But the consciousness is in the encoding of the rock structure not each individual rock. Just like molecules taken out of the brain are not conscious, or the brain itself, when dead with no one to "move the rocks" (send neurons electric signals) is not conscious. It's the structure that is encoding the consciousness.

[Edited after reading the XKCD strip, in which by "throwing rocks on sand" means immitating a turing machine with rocks and sand].

Or apply modus tollens and infer our consciousness is not reducible to our meat machines.

Nobody said it is: it's in the interconnections and stored info inside the meat machines. A T-bone is also meat but without consciousness.

Except if you think it's in an external, not meat/neurons/info soul.

Substrate independence - whether it's connections or something else - is a very strong claim that should be subjected to empirical testing before being accepted. (And IMO is almost certainly wrong for good practical reasons that have nothing to do with souls.)

Of course you can't test it empirically, because there's no empirical test for subjective experience. You can only test for the appearance of subjective experience - specifically for external behaviours, loosely defined, which might be considered to imply subjective experience to you as an observer.

This isn't a solvable problem, and Bostrom is begging the question by trying to hand wave it away with a sneaky "If we just accept this premise..."

> good practical reasons

The only reasoning I have seen on this subject so far has been "people are special" or "God made me". Sometimes the arguments are dressed up more. Do you have a good source that has a different type of reasoning?

Edit:

> You can only test for the appearance of subjective experience

Yes, and we solve this in people by assuming that if they sound, act and look like a person they are a person. I've never seen a compelling argument why it is somehow wrong to apply this principle to everything else, except of course the huge trouble philosophers go to to then argue what exactly needs to be looked at before we are satisfied. For some reason the Turing test is not adequate, but to my mind it's only not accurate if you believe that you are somehow special and that there is more to you than what can be observed.

> For some reason the Turing test is not adequate

Let's say I build a chatbot with a predetermined database of perfect responses to every possible string of words. This is quite clearly possible in theory, given a database with infinite room. This chatbot would pass the touring test, even though all it's doing is retrieving prewritten sentences. Is it conscious?

Ergo, it's quite clearly possible to build an automaton which acts like a human, but actually has no clue what it's saying. Which means the touring test isn't adequate.

---

Here's another take:

You have no proof that anyone in the universe is conscious other than yourself. However, given that you were born in the same way as everyone else, and you're conscious, everyone else probably is too. Why would you be so special?

Well, a computer program was not born in the same way that you were, so that assumption goes out the window.

>Let's say I build a chatbot with a predetermined database of perfect responses to every possible string of words. This is quite clearly possible in theory, given a database with infinite room. This chatbot would pass the touring test, even though all it's doing is retrieving prewritten sentences. Is it conscious?

For all intents and purposes, yes. Though without actual consciousness it would be impossible to build it.

>Well, a computer program was not born in the same way that you were, so that assumption goes out the window.

Well, a computer airflow program was also not created in the same way as actual air, but it can simulate it to the degree that we use it to build actual planes.

A computer program might not have been 'born' in the same way that we are, but we can make it simulate the "way that we are" (giving enough computer power, down to the atom or quantum element). So?

You have succeeded in making the "people" are special argument twice. I think that sort of underlines my point. Human supremacist arguments are not convincing to me.

>Substrate independence - whether it's connections or something else - is a very strong claim that should be subjected to empirical testing before being accepted. (And IMO is almost certainly wrong for good practical reasons that have nothing to do with souls.)

Care to mention any of those "good practical reasons"?

Penrose gave it a try (though even there, there was nothing about how it can't be modelled with code for example, in which case, rocks are fine too for a simulation a la XKCD, just not as a direct substrate for an Android).

>Of course you can't test it empirically, because there's no empirical test for subjective experience. You can only test for the appearance of subjective experience - specifically for external behaviours, loosely defined, which might be considered to imply subjective experience to you as an observer.

It's not really that we can't test it, it's more that we can't measure it. But we can, and do, test it, e.g. with psychological tests, the same kind we give to people to understand different aspects of their personality (which we also can't just measure with some equipment or simple measurement). If it quacks like a duck, and looks like a duck, and cries like a duck, and has other sentimental duck qualities, it's a duck. We only accept the subjective experience of other people from its appearance (their actions etc) too, anyway.

There’d be no reason to suspect anything could be conscious, if it weren’t for the observation that a pile of meaty goo can be conscious.

Given that meaty goo can be conscious, and assuming that this arises from physical properties rather than some form of actual magic, it is reasonable to guess that the consciousness arises from what the goo does rather than from other properties such as pinkness or gooiness, so if we can cause another physical system to do the same things then it’s likely to have the same subjective experiences while doing it.

I'm sorry if I repost this link too often but I find this take on consciousness just too interesting. Read for yourself - the author is the (in-)famous machine learning researcher Jürgen Schmidhuber:

https://www.reddit.com/r/MachineLearning/comments/2xcyrl/i_a...

There's a problem with that description which the author implicitly acknowledges with his use of language:

> "...the agent could be called self-aware."

"Could be called" being the key phrase. Yes, you can call such a model self-aware, because indeed it incorporates a model of itself so it is self-aware in a very literal sense.

However, there's no reason to believe that a model with the ability to represent itself would have conscious experience of the kind humans do. Why should we expect it to be any different from any other ML model or computer program, just mechanically executing its program without any conscious experience? What stops it from being, basically, a machine version of a philosophical zombie?

This is the hard problem of consciousness in a nutshell, and the Schmidhuber comment doesn't address it in any meaningful way.

I thought i read something from him which was more esoteric when he was describing a soul but can't find anything.

Your link is quite technical.

Thanks for this, it's new to me and that entire ama is fascinating.

edit: Out of curiosity, why would one consider him infamous?

He's known by many in the AI/ML community as someone who likes to slay down claims of novelty and likes to dig up connections to decades old ideas (which heavily include his own works). He's controversial, meaning that many people like him and many dislike him, few are neutral.

Perhaps look up the case with Ian Goodfellow's work on GANs. Schmidhuber said the GAN paper doesn't give sufficient credit to his earlier work on predictability minimization and was very vocal about this also in conferences (interjecting etc.).

>Out of curiosity, why would one consider him infamous?

IIRC claims were made that LeCun etc. were unfairly credited for work pioneered by him. As in, he pretty much invented modern ML but wasn't credited at all.

I love the bunch of rocks example and think about it a lot.

If a simulation cannot produce consciousness, what is special about life that makes it unrepresentable with physics? Where is the boundary between a conscious system and a non-conscious one?

Conscious minds certainly seem bound by their physicality, you can figure out how feelings and experiences are represented molecularly and electrochemically and affect those feelings and experiences experimentally. So far we've not found anything that cannot be broken down into physics (not to say we won't), but if we assume there's nothing else _but_ physics – then I think we're left with:

- The physics consciousness relies upon is fundamentally uncomputable

- Consciousness _is_ computable and all the weird consequences that follow (like sand computers) are true

I lean towards the second option, but I struggle to understand it in any intuitive sense.

What are your thoughts on why a simulation cannot be conscious?

I think you're missing another possibility (and some other posters are too): maybe it is the case that a simulation can produce consciousness, by simulating a brain for example, but that it takes a sufficiently powerful model of computation to do so.

So, consciousness being computable may not necessarily imply that any given computer could do so. One potential way for this to be the case is if consciousness is tied to some form of hypercomputation (https://en.wikipedia.org/wiki/Hypercomputation), which can provide outputs that are not turing-computable. This would mean that a traditional computer couldn't simulate consciousness, but that another hypothetical machine might be able to.

Yes, exactly.

Bostrom is assuming there's a thing called "computation" which is somehow a fully-specified solved problem. (It isn't).

Worse, he's also assuming that this "computation" can somehow simulate something called "consciousness" - which unfortunately for him, is nowhere near to having the formal rigorous definition that "computation" as we know it today would require.

And finally after all that, you get something which is subjectively indistinguishable from life itself.

Well - obviously. If you don't define any of your terms and assume your argument is correct because you don't show your working or get into specifics, you can argue whatever you like.

> what is special about life that makes it unrepresentable with physics

absolutely nothing

> The physics consciousness relies upon is fundamentally uncomputable

This would be "huge if true", but I don't think it is. At least I haven't seen any indication of why it would be.

> Consciousness _is_ computable and all the weird consequences that follow

I think this is true, and I think the weirdness is a result of misunderstandings on our part (we are notoriously bad when reasoning about systems that contain us in some way).

> What are your thoughts on why a simulation cannot be conscious?

I think this whole consciousness confusion is due to mixing levels of abstraction on our part. Replace consciousness with other higher-level cognitive or social functions, like "love" or "patriotism", and see how silly it sounds. Is "love" computable? Can "loyalty" be reduced to physics? Yes, but the concept makes sense on a whole different conceptual level, which would be silly to reduce to, say, quantum physics, and is much better to reason about at the right abstraction level.

I lean toward the first option for the following reason: I doubt that physical processes can be simulated precisely by a Turing machine.

For example, if I'm living in a simulation and I move my hand an inch to the left, it occupies an uncountably infinite number of intermediate positions during that motion. A Turing machine can't calculate or even represent all of those positions, since the set of Turing-computable numbers is only countably infinite.

Of course, what I've written above assumes that spacetime is continuous. Is it?

That can be tested by experiment:

https://physics.stackexchange.com/questions/9720/does-the-pl...

According to that link, spacetime shows no sign of discretization down to fourteen orders of magnitude below the Planck length. (The Planck length itself is so small that if an object that size were expanded so as to be a millimeter across, then a proton, at that scale, would be larger than the distance between the sun and Alpha Centauri.)

That suggests to me that while Turing machines can provide useful approximations to the behavior of physical systems, they can't represent or simulate them precisely.

I don't think there are an infinite number of positions. The Planck length [1] essentially defines the smallest unit of length. Similarly, Planck time [2] does the same for time.

[1] https://en.wikipedia.org/wiki/Planck_length

[2] https://en.wikipedia.org/wiki/Planck_time

Your first reference states that the Planck length isn't the smallest unit of length in standard physics: "The Planck length is sometimes misconceived as the minimum length of space-time, but this is not accepted by conventional physics, as this would require violation or modification of Lorentz symmetry." It then states that alternative theories do allow for a minimum length.

Experimental evidence (the Fermi satellite data discussed in the link in the grandparent comment) seems to support standard physics.

Computation does not need to be digital. An analog computer could represent positions down to arbitrary precision, though accuracy would still be limited by the information input.

> If a simulation cannot produce consciousness, what is special about life that makes it unrepresentable with physics?

Authors can write stories with believable worlds and compelling characters. Given infinite time, an author could detail how a character would react to every possible situation. But no matter how much they wrote down, their words would just be words. There isn't some breaking point where the author writes enough to create a pocket universe.

Why are turing machines different?

Just because the universe is governed by physics does not mean that physics are sufficient to construct a universe. I expect that "physicality" is a real property of some kind, which doesn't appear just because you described it accurately.

So much of this is guesswork, I may well be wrong—but it strikes me as the most logical conclusion.

It's just semantics. A brain is just a "simulation" of a consciousness. Unless you want to bring religion / spirituality into the discussion, there's absolutely nothing to suggest that there's something magical about the human brain that allows it to support a consciousness in a way that a hypothetical man-made computer cannot.

I see no reason that bringing in simulation should be considered more reasonable than bringing in religion.

In fact, it pretty much boils down to the same thing, as far as I can tell. You have a simulation? Someone set it up. He/she/it is the creator of this universe.

[Edit: I should say that a simulation winds up being the same as a religion, but I'm not sure that a religion necessarily winds up being the same as a simulation.]

That's working on an issue from two different directions. We work towards considering a simulation probable because of what we ourselves are working towards creating.

Stating the cause/creator first and looking for evidence of it is not very reasonable.

We are so far from being able to do any such thing that I take issue with the phrase "working towards creating".

Magical in the sense how radioactivity was magical before it was known. We might be missing something important in our knowledge of universe and once we find it, things might start making sense.

What if our tree of life’s brain is utilizing a type of Quantum Computing?

Considering how much effort is required to just get a handful of qbits in the lab, I find it highly unlikely that we've got heretofore undetected qbits hiding in our (and, presumably every other animal's) nervous systems.

That might be argument like when aluminum was the most expensive element and rich kings had aluminum roofs on their castles, because nobody was able to manufacture it cheaply. We might just be clumsy...

This school equates simulation with being, so that if consciousness is simulate-able then it exists as consciousness. The interpretation of simulations is as far as we can tell in the mind of a beholder, but simulation hypothesis has come to accept that anything which can compute, can create realities on equal footing with the common reality which our consciousness seems to help connect us to.

On one hand it is a wonderful outlook where computation and imagination creates universes as real as our universe feels to us (or rather as real as each creatures individually experienced universe seems to them). On the other hand, it does concern me that our own and fellow creatures experiences are perceived by this school as formally attributable to controllable processes, distinguished in "reality" only by beholders individuated interpretations, be they someone, a convincing replication of someone, or anything really which any individual decides to treat as conscious or not.

Consciousness doesn't have to be all encompassing. Might be a small rider on a gigantic elephant, i.e. just a small part of the simulation - https://en.wikipedia.org/wiki/Jonathan_Haidt#The_Elephant_an...

If not, why can a brain with a bunch of neurons and chemicals?

Either: - It can, and so can rocks. - It can, but there's something else unknown that's necessary to generate consciousness. - It can't, consciousness doesn't exist.