People in this debate miss a subtle issue, and it causes no end of confusion. The subtle issue is this. There is a difference between (1) accidentally building a computer which happens to think, and (2) intentionally building a computer which thinks, and knowing that it thinks.

Obviously (1) is possible, because for any particular configuration of atoms, it is theoretically possible for someone to assemble atoms together in that configuration. I could randomly put atoms together and, by sheer luck, assemble a living, breathing Abraham Lincoln. But since it happened by random chance, I wouldn't know for sure that this was really a living, breathing Abraham Lincoln. I might observe it for awhile and notice that, golly, it sure does seem to be a living, breathing Abraham Lincoln. But I could never rule out the possibility that it only looks like that initially and that it'll eventually collapse or fall into some infinite loop or something.

Intentionally building a machine and KNOWING that it has such-and-such properties of a thinking machine, is much more interesting. Such KNOWING is, itself, an act of thinking, and so there arises an interplay between the thinking of the created machine, and the thinking of the creator. Then you can start asking questions about whether the created machine, being a thinker, could also itself intentionally create another thinking machine, and so on.

If you're tired of the endless non-productive debate about whether thinking machines are possible, and you'd like to go to the next level, and take steps toward actually quantifying things, I'd invite you to read some papers I've written on the subject. To lessen the learning curve, here are some very approachable slides, and if you like those slides, you can click through to one of the papers. https://semitrivial.github.io/MeasuringIntelligence2019.pdf

Isn't this the same as the "problem of other minds"? How do I know that YOU mr xamuel are actually sentient?!

If we get to the point where a machine demonstrates "sufficiently" complex behavior, society will quickly accept them as sentient. This is something we are primed to do. Hell, people ascribe all sorts of intentionality to their dog.

After a few years, society at large will accept these machines as sentient, and it'll be the same philosophers arguing about "but are they REALLY sentient" that have been arguing the problem of other minds since ... Gorgias in 4th century BC?

You raise a really interesting point / distinction. I'm just arguing that society will race past this question without hesitation.

Bonus points for mentioning Gorgias :)

Re: wouldn't society race past this question without hesitation [if someone got lucky and produced what appears to be strong AI by just randomly throwing atoms together]?

Yes, and in fact some people would argue that is precisely what has in fact happened, the random atom-throwing-together process being called "evolution". And as you point out, philosophers would continue to debate about "is it REALLY strong AI?", just as nowadays philosophers debate about whether humans are machines or not.

These hypothetical debates will always exist, nothing will ever change that, not even if someone randomly creates a true strong AI by sheer luck.

My point is that there's a whole other half of the field, which ISN'T doomed to perpetual fruitless philosophical debating. The question of whether intelligent machines can exist is one that is doomed to fruitless debates forever. But the question of whether we can build specific machines and know with mathematical certainty how quantitatively intelligent they are---that is a much more fruitful area where progress can be made, theorems can be proved, mathematics can be applied, etc.

Unfortunately there's no agreement that we can actually quantify intelligence in a useful way, and believing that "ability to reason" and intelligence are synonymous, never mind evidence of sentience, is clearly incorrect.

Whatever human intelligence is, it has multiple dimensions, and being "able to reason" - whatever you assume that means - is a tiny element of a bigger picture.

It's practically a universal law of AI that proponents of AI inevitably understand AI in terms of their own niche interests - whether those are chess, go, e-sports, music, or driving. Only a field like CS could conceive of a test where the ability to emulate a conversation by typing lines of text on a terminal might be considered an indicator of operational sentience.

In reality all but the absolutely dumbest humans can operate a body fairly effortlessly, feed themselves and keep themselves clean, express characteristic preferences and act on them with agency, read and respond to human expressions and emotional cues, communicate internal emotional and psychological states verbally and non-verbally, improvise solutions to simple problems, maintain a fairly consistent emotional and psychological outlook that nonetheless develops over time, and make occasional surprising and unexpected - but not inhumanly weird - statements, observations, and actions.

How much of that is considered "being able to reason"?

If there's a way to quantify ability in all of the above, I'm not aware of it.

As a whole, an AGI is so general that we'll probably never be able to quantify its intelligence level. But we can quantify the intelligence of various "cross sections" of an AGI. As an extremely simple example, whatever the heck an AGI is, presumably an AGI can play chess. Therefore, we could instruct an AGI to compete in chess games and we could compute its Elo rating. This wouldn't be a measure of the AGI as a whole, but it would be a measure of the AGI as a chess-player.

More generally, we can consider the AGI to be a reinforcement learning agent: not that the AGI reduces to nothing but a reinforcement learning agent (that would be ridiculous), but rather, whatever the heck an AGI is, an AGI would certainly be capable of performing in reinforcement learning environments. Various methods have been proposed to measure RL agent intelligence, ranging from more practical to more philosophical.

Again, an AGI can be considered as a knowing agent, and identified with the formal set of mathematical theorems it knows in some language. Again, this isn't to say that's ALL the AGI is---that would be absurd. But it is one facet of the AGI. Whatever the heck the AGI is, it presumably knows some set of mathematical theorems in any given language (possibly the empty set, possibly a non-consistent set, etc.) There are various ways of classifying and even quantifying sets of mathematical theorems in a given language. See the slides I linked above for some details on this particular cross section.

> Only a field like CS could conceive of a test where the ability to emulate a conversation by typing lines of text on a terminal might be considered an indicator of operational sentience

Turing first wrote of that test in 1950. I don't think text was the primary way computers were interacted with back than, so if anything, it might be more plausible that cause for coming up with that interface was precisely because it's something that human-like intelligence is well-suited for rather than the reverse.

> Only a field like CS could conceive of a test where the ability to emulate a conversation by typing lines of text on a terminal might be considered an indicator of operational sentience

The point of using the terminal is to double-blind the role of the Judge. They have to decide whether they are communicating with a person or a machine - doing it face to face would rather give the game away.

When people imagine what a Turing Test conversation would look like, they frequently underestimate the conversation. I find Dennet's example of an imaginary Turing Test from Consciousness Explained to be a good counterexample:

Judge: Did you hear about the Irishman who found a magic lamp? When he rubbed it a genie appeared and granted him three wishes. “I’ll have a pint of Guiness!” the Irishman replied and immediately it appeared. The Irishman eagerly set to sipping and then gulping, but the level of Guiness in the glass was always magically restored. After a while the genie became impatient. “Well, what about your second wish?” he asked. Replied the Irishman between gulps, “Oh well, I guess I’ll have another one of these.”

CHINESE ROOM: Very funny. No, I hadn’t heard it– but you know I find ethnic jokes in bad taste. I laughed in spite of myself, but really, I think you should find other topics for us to discuss.

J: Fair enough but I told you the joke because I want you to explain it to me.

CR: Boring! You should never explain jokes.

J: Nevertheless, this is my test question. Can you explain to me how and why the joke “works”?

CR: If you insist. You see, it depends on the assumption that the magically refilling glass will go on refilling forever, so the Irishman has all the stout he can ever drink. So he hardly has a reason for wanting a duplicate but he is so stupid (that’s the part I object to) or so besotted by the alcohol that he doesn’t recognize this, and so, unthinkingly endorsing his delight with his first wish come true, he asks for seconds. These background assumptions aren’t true, of course, but just part of the ambient lore of joke-telling, in which we suspend our disbelief in magic and so forth. By the way we could imagine a somewhat labored continuation in which the Irishman turned out to be “right” in his second wish after all, perhaps he’s planning to throw a big party and one glass won’t refill fast enough to satisfy all his thirsty guests (and it’s no use saving it up in advance– we all know how stale stout loses its taste). We tend not to think of such complications which is part of the explanation of why jokes work. Is that enough?

Dennett: “The fact is that any program that could actually hold up its end in the conversation depicted would have to be an extraordinary supple, sophisticated, and multilayered system, brimming with “world knowledge” and meta-knowledge and meta-meta-knowledge about its own responses, the likely responses of its interlocutor, and much, much more…. Maybe the billions of actions of all those highly structured parts produce genuine understanding in the system after all.”

I'm not sure (1) is 'obviously possible'. Supposing there's a finite number of atomic configurations that produce Abraham Lincoln, there's still an infinite number of atomic configurations so you're probability is strictly zero, no? It's not even an 'age of the universe to achieve' type event. And that assumes that your random throwing together of atoms includes the physical preconditions to create an Abraham Lincoln. For instance, if you were sitting in the middle of a black hole randomly assembling atoms I'm pretty sure it's physically impossible to create an Abraham Lincoln.

> I could never rule out the possibility that it only looks like that initially and that it'll eventually collapse or fall into some infinite loop or something

I don't put much stock in that possibility. Physicalism essentially closes the door on p-zombies, no?

Until we solve the hard problem of consciousness, the door on p-zombies is wide open. It's arrogant to assume that just because you are conscious, everyone else must be conscious like you. Maybe consciousness is caused by a particular benevolent parasite living in your gut, and whoever lacks that parasite lacks consciosness.

> It's arrogant to assume that just because you are conscious, everyone else must be conscious like you.

It's far more arrogant to assume you're the odd one out. There's a reason we never use that kind of thinking. The following reasoning is clearly absurd:

I think I'm immune from radiation sickness. I know we have no evidence that anyone in history has been immune, but I've never been exposed, so I'm going to assume I'm immune.

> Maybe consciousness is caused by a particular benevolent parasite living in your gut, and whoever lacks that parasite lacks consciosness.

You could go with that explanation, but it's far more complex than necessary, which is a good indicator that it's not true.

A physicist won't be able to disprove the suggestion that gravity will only exist for as long as the Invisible Pink Unicorn wills it to exist. So what?

We can be pretty confident that consciousness arises from the normal activity of the brain. We can temporarily shut it down with anaesthetics, and see the diminished activity in parts of the brain.

>It's far more arrogant to assume you're the odd one out.

I didn't say you should assume that either. You shouldn't assume either one until you have some reason to do so.

>A physicist won't be able to disprove the suggestion that gravity will only exist for as long as the Invisible Pink Unicorn wills it to exist. So what?

If I had some way of knowing which people are conscious, and I observed that everyone currently living and everyone in the past was always conscious, then it would indeed be silly for me to suggest that tomorrow many non-conscious people will be born --- as silly as your Invisible Pink Unicorn example. But the fact is, I do NOT know that anyone other than myself is conscious. I assume they are for the sake of adjudicating ethical dilemmas, but I only actually have that empirical data for myself.

>We can temporarily shut [consciousness] down with anaesthetics, and see the diminished activity in parts of the brain.

Here you seem to be assuming consciousness is identified with certain brain activities, but that's begging the question. For example, that would rule out computer simulations from being conscious, since you couldn't detect said brain activity in said computer. But it's far from settled whether or not consciousness is impossible in a computer simulation, in fact, many people think it IS possible. So unless you know something a lot of experts don't know, there isn't enough evidence to equate consciousness with certain brain activity.

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim" - Edsger Dijkstra

Put more simply: if we assume the human mind to be material, and deterministic, then by definition it is possible to replicate it in a computer (at least one with infinite resources).

At the same time, contrary to the popular media narrative, today's "AI" is still lightyears away from that.

Strongly agree. Banks expresses those points well.

The piece fails to account for that we already know that it's possible for physical entities to think, and to make independent decision. We know this because human beings exist.

No good reason is offered to justify the idea that the difference in intelligence between humans and computers is a difference of category, rather than merely a difference of (ever shrinking) degree.

From the article:

> The intellectual incoherence of believing that computers can think or make independent decisions is self-evident

Nonsense. This is a very strong claim about the universe, or rather, one of two equally silly claims:

* That general intelligence relies upon the process of evolution by natural selection, and that it can never even in principle arise from a process of deliberate design

* That general intelligence relies upon a particular substrate, specifically neurons, and that it cannot be achieved with other substrates, such as transistors

Such a claim cannot be justified with a hand-wavey appeal to intuition, or with arguments about moral accountability, which are all the author has to offer.

Also, this next quote strikes me as misleading. Perhaps someone more knowledgeable than me can confirm/disconfirm my take:

> chess prowess depends on players’ ability to recognize general situations that are in some sense “like” ones they’ve seen before, either over the board or in books. Again, the computer largely sidestepped what is most significant for humans, and hence for the game, by analyzing every position from scratch, and relying on speed to make up for its weakness at gestalt pattern recognition.

That may be true of chess, but what about Go? As I understand it, the reason computers can now play Go effectively (i.e. Alpha Go), is that a breakthrough was made in discarding unpromising subtrees of the game's possible progression, allowing effective moves to be selected relatively efficiently. That's not brute-force. That's precisely the kind of clever trick we'd expect from a skilled human player.

The whole piece strikes me as poorly considered. The thoughts of Dan Dennett are far more worthwhile. If you've read this far, you may enjoy a quick read of the Wikipedia article on Dennett's intentional stance. [0]

Dennett also makes the relevant observation that mankind is constantly creating machines with general intelligence and moral accountability: other human beings. This ties back to the article's points about moral accountability.

[0] https://en.wikipedia.org/wiki/Intentional_stance

>"that there is some vital field or other presently intangible influence exclusive to biological life"

There is. The ability to instantiate one specific abstract into a physical item. An abstract can for example be the idea of triangularity and the physical substrate will be the brain.

No AI running on a typical Turing machine can do that. For any idea you believe you have instantiated into the machine you can come up with an alternative interpretation of what you actually have instantiated.

This means that any physical system alone underdetermines the abstract thing it is supposed to be an instantiation of. You can for example likely find a pattern within the atoms of the wall behind you that could be said to be an instantiation of the quicksort algorithm. Or it's just a wall. The physical item itself is silent on the matter.

This doesn't make any sense to me whatsoever.

The phrase you've quoted is talking about whether there is something supernatural in the construction of a brain that gives rise to the consciousness, e.g. a soul.

The point being that if the brain is "just" bioelectrical hardware, there is very little reason to think we can not eventually reverse engineer it. But if there's a "secret black box" element to consciousness that exists outside the real of the possibility for us to examine and recreate it, then that is a separate issue.

"Intelligence" is the "easy" part - there's no reason to think we can not eventually simulate reasoning to a sufficient fidelity; dealing with symbolic manipulation of abstractions is certainly not impossible.

Consciousness is the big question, because we don't know what it is.

If the brain is just bioelectrical hardware then there is nothing worth reverse engineering, nor is there any worth to anything at all.

You can simulate digestion with a turing machine all you want, but that simulation wont digest any food. Likewise you can simulate the neurons in a brain but that simulation can not be an instantiation of the concept of triangularity.

Of course there's something worth reverse engineering, because in that case doing so would allow us to create consciousness as well as transfer and copy it. It would hold the key to at least a form of practical immortality.

If the brain is not just bioelectrical hardware, then we have the problem of whatever it might "interface with" can be influenced and copied, and we don't even know if there is anything to interface with and copy.

> You can simulate digestion with a turing machine all you want, but that simulation wont digest any food. Likewise you can simulate the neurons in a brain but that simulation can not be an instantiation of the concept of triangularity.

This is an utterly meaningless paragraph. If we build a robot we most certainly could build one that actually digests food. To my knowledge we can't yet do so without actually losing energy rather than gaining it, but that is an issue with the efficiency of the engineering, no fundamental blocker. If in software, it wouldn't digest any food, sure, but it also would have no need to, and that has no parallel to the second part of your paragraph.

The "concept of triangularity" can certainly be simulated in an abstract manner. It doesn't even take a complex system. AI doing symbolic manipulation of abstract concepts is one of the earliest things we tried. It's not the hard part. The hard part is actually reasoning about those concepts, inferring them, being able to act on them and talk about them. But that too is orthogonal to solving the issue of what consciousness actually is.

Afaik we can throw bacteria at it and then use something similar to fuel cells to digest their products. There are already robots that you can feed by placing apple slices at the right location on them. Methane fuel cells driven by methane from anaerobic bacteria is not hard. The digestion is used for a long time in waste water treatment, and the fuel cells are good enough to be useful for a few years now. It's just that those fuel cells are _expensive_, and the system power density is quite meager, all considered. It also lacks an immune system that keeps the bacteria in check, so it's not that hard to poison (still quite resilient though).

No, it would not allow you to create consciousness, it would simply mean that there is no such thing as consciousness. eliminative materialism would be true. there would be no 'you' to live forever.

Us not knowing exactly how something works has never stopped us before from looking into it.

With "a simulation of digestion" I specifically mean a software simulation, not a hardware simulation. I assure you I had a meaning with the text I wrote, it is not meaningless. The meaning is for me to say, your interpretation of it (and this is the point) underdetermines the meaning of it. You fully determine what i meant simply from the characters on the screen. Physical instantiations of abstract items work the same way. An item needing to do something is irrelevant of it being able to, a software simulation of food does not actually digest food. A software simulation of abstract concepts do not actually instantiate them.

I am repeating myself again but any AI system you believe to have instantiated the abstract concept of triangularity is just an interpretation on your part to having done so, you might as well interpret it to simply being a room heating device or the implementation of some other algorithm that also matches the behaviour.

> No, it would not allow you to create consciousness, it would simply mean that there is no such thing as consciousness. eliminative materialism would be true. there would be no 'you' to live forever.

Do you believe you are conscious? I believe I am conscious. If the brain is purely bioelectrical, then a purely bioelectrical system at least is able to reproduce an entity that believes itself to be conscious.

Whether there is a meaningful distinction between that and "actual" consciousness is something we simply don't know, because we don't have a way of measuring whether or not another entity actually is conscious, whatever we take that to mean.

Maybe that belief that I am conscious is flawed and merely an illusion, but if copying my mind could allow that illusion to persist forever, then my illusion of consciousness would still be quite pleased with that outcome. That is quite enough for me.

> I am repeating myself again but any AI system you believe to have instantiated the abstract concept of triangularity is just an interpretation on your part to having done so, you might as well interpret it to simply being a room heating device or the implementation of some other algorithm that also matches the behaviour.

Here is an implementation of the abstract concept of triangularity that does not involve an interpretation of what it means or an instantiation of the concept, merely a symbol that describes the abstract concept as it is:

    thought = :triangularity

I define the symbol :triangularity to refer to the abstract concept, so it does. I can manipulate that concept with or without making any interpretation as to what the concept actually means.

Symbolic manipulation of abstract ideas is trivially easy. The idea that we can't is ludicrous. The challenge comes when we try to interpret and reason about specific concrete interpretations, not in holding an abstract concept - we've known how to communicate abstract concepts as data without instantiating an interpretation since at least the moment we could write.

You're attacking AI on the basis of one of the problems that are easiest to solve. Symbolic manipulation of abstract concepts with computers in a meaningful form dates back to at least the 50's. The ease with which we can do that was part of much of the early misguided hope of a rapid revolution in general artificial intelligence.

> Maybe that belief that I am conscious is flawed and merely an illusion

That's the Dennett nonsense. Of course our sense inputs are an illusion, it's that there is anything being perceived at all that is the mystery! :

DENNETT: So you see, Socrates, consciousness is an illusion.

SOCRATES: I see, but I need to clarify something. What do you mean by illusion?

DENNETT: An illusion is an appearance contrary to reality.

SOCRATES: And what is an appearance?

DENNETT: An appearance is something that is apprehended by an observer.

SOCRATES: So consciousness is something that is apprehended by an observer which is contrary to reality.

DENNETT: But that cannot be, because then the observer would have to be conscious.

SOCRATES: Therefore, consciousness cannot be an illusion!

This makes no sense to me.

The point I was making is that we do not know what consciousness actually is. Until we do, we have no way of telling whether my impression of my own consciousness has any reasonable relation to the reality of it. It is meaningless to try to determine whether or not it is an illusion without first having an understanding of what consciousness is.

More directly to the argument I was addressing, the point is that whether or not the commenter I replied to is right and there'd be no "me" that could live on forever, that is really rather irrelevant to the "me" that believes I am conscious.

In other words: While I need clarity in what consciousness is to be able to even begin to address whether or not what I have is truly consciousness, it really doesn't matter to the question of whether or not whatever I am sees value in a process that'd let me live on.

You don't have an impression of your own consciousness (your conscious awareness) to begin with. You just perceive stuff. How exactly could you not have conscious awareness?

We can't even meaningfully answer that question without an understanding of what consciousness is first.

Consciousness appears to us as if we are distinct entities, with a spatial and temporal existence, operating on sensory input and memory. We don't know if any of that is true - we don't know if there is any persistent entity, whether spatially or temporally.

I'm not saying there's "nothing" that we could label consciousness. I'm saying we have no way of suggesting whether or not our idea of ourselves as an entity that experiences events and acts on them (even just mentally) is meaningfully connected to reality.

My point in making that argument is that it doesn't matter - absent any better information, the only thing that makes sense for us is to act on the basis that our consciousness is real, and interact with people on the basis that they're real and conscious, because it doesn't matter whether or not it is an illusion if we can not "see past the veil".

This is incidentally also the classical response to philosophical idealism: Sure, maybe all our senses lie to us, but since everything acts as if we're in a materialist world, it doesn't matter.

It remains possible that at some point we find something that lets us lift the veil, and peer out into something "outside" our present reality. But that possibility does not change anything until and unless it becomes reality.

It does not follow that " a purely bioelectrical system at least is able to reproduce an entity that believes itself to be conscious" simply because you believe yourself to be conscious. A human simply being purely bioelectrical is the very thing in question. Further, a belief, as somethign abstract, is not something that a bioelectrical system could in principle capture, and that we do know, in such a scenario there is no 'you' to be 'pleased' any more than a rock could be.

"thought = :triangularity"

That is simply some LCD pixels on my screen. What you define it to be has absolutely no bearing on the physical implementation of the pixels. You could likewise define it to be a partial sentence on 3d polygonal mesh generation from a math phd thesis or any number of other interpretations. Any interpretation you make is IN YOUR HEAD. Not in the physical instantiation (the LCD pixels or memory the transistors of the RAM).

The problem here is that you don't understand the point.

> It does not follow that " a purely bioelectrical system at least is able to reproduce an entity that believes itself to be conscious" simply because you believe yourself to be conscious

You are right, that does not follow. It does however follow that if we are purely bioelectric systems, and that is sufficient for us to believe ourselves to be conscious, then it is reasonable to presume that a clone of such a system will presume itself to be conscious. That is the claim I was making.

> A human simply being purely bioelectrical is the very thing in question.

You cut of "If the brain is purely bioelectrical" from the part you quoted above, and so what you attacked above was a strawman of your own making, and not the argument I made.

> That is simply some LCD pixels on my screen. What you define it to be has absolutely no bearing on the physical implementation of the pixels.

That is entirely irrelevant. It is a representation of an implementation of an symbolic representation of an abstract thought in a programming language that runs on a Turing machine (it is valid Ruby). As such it is proof that a turing machine can represent abstract thought.

> The problem here is that you don't understand the point.

We clearly do not understand each other, but so far I have seen nothing to suggest that you have a meaningful argument. You're trying to create a distinction between the abstract and an instantiation of the idea to create a separation that makes representing the abstract notion in a computational device impossible. But representing abstract thought in symbolic form is something humanity has been capable of four hundreds if not thousands of years at a minimum. The point is that abstraction and symbolic representation does not get any less real because you invent additional intermediate steps.

Sure, what you read appeared to you as some LCD pixels on your screen, but that does not make it any less a symbolic representation of the abstract idea, rather than an interpretation of the idea. If you fail to understand it and instead interpret it as a sentence on 3d polygonal mesh generation is irrelevant - it does not change the abstract notion, any more than misinterpreting a thought changes the original thought.

You argued we could not represent anything but an interpretation of an abstract idea. The above is proof this is not true, because any abstract idea can be exchanged for a symbol representing that idea, and the symbol can be accurately reproduced.

This is a foundational aspect of communications, of symbolic logic, of linguistics, and a whole range of other scientific disciplines, because it is such a central aspect of the ability for human discourse about abstract ideas.

If you want to attack AI there are many angles to take - we do not know whether or not consciousness requires some form of dualism for example, hence my constant qualifications ("if the brain is purely bio-electrical") about it. That's a weak spot.

But our ability to represent abstract ideas in computation is not one of the weak spots. It's one of the easiest parts of AI, or indeed of any computation.

The understanding of how to represent abstract notion by symbolic substitution is old enough that you see that transition happening in language going back thousands of years, when concrete concepts starts being used as symbols of abstract ideas.

That does not follow either for if we are purely bio-electrical then we can not have beliefs.

Strawman? What?

Programming languages running on turing machines are abstract entities, they only exist in your mind, not in reality. The physical machine just shuffles electrons around.

Abstract Thought -> Symbolic Representation -> Physical Implementation.

The right hand side is not the left hand side. Any right hand side implements an infinite left hand sides and there is no way for you to pick out the correct one simply from the physics. I have never written the word representation before in this exchange, yet you claim i have. Stop doing that.

You wont get it. This is a standard part of philosophy of mind. Look it up.

> If the brain is just bioelectrical hardware then there is nothing worth reverse engineering, nor is there any worth to anything at all.

may I introduce you to our local nihilist club

If we can make a faster or parallel perfect copy that would be rather valuable - uploading, copying and "fork/join" of minds would be rather useful.

not nihilism, materalistic eliminativism - just happens to have nihilism as a consequence.

> You can simulate digestion with a turing machine all you want, but that simulation wont digest any food.

You could digest simulated food.

You could also build a machine stomach to digest real food.

If you found out you were a simulation you’d be looking pretty dumb right now

> then there is nothing worth reverse engineering, nor is there any worth to anything at all.

Non-sequitor. You are certain to die, so there's no point in continued breathing?

It is not a non-sequitor, it follows from the concepts, something being worth doing is not a part of physics/math. If math/physics is all you have then you have no "worth doing".

If it turns out we're just simulated entities, does that suddenly make your life not worth living?

Why would it change anything?

if we are simulated entities then there is no "we". 'we' do not have existence. that would indeed make life not worth living, on account of there not being a life there in the first place

If we are simulated entities, then the fact that I at least live a pleasant life would show that it is perfectly possible to have a pleasant, worthwhile simulated life.

no it would show that you are not simulated

It might, if it turns out there's an opportunity for a reset.

> You can simulate digestion with a turing machine all you want, but that simulation wont digest any food.

Because food is a physical item. Ideas and abstract concepts are not and can be communicated in a format that's fully compatible with a simulated consciousness. There is no reason that wouldn't be possible aside from mechanisms that are, or at least appear, supernatural.

You're talking about an ability, Iain Banks was talking about some unknown bit of physics, a field or somesuch.

From what do you posit this ability to "instantiate one specific abstract into a physical item" comes from, and why do you think only wetware can do it?

Reading it again, I think you might be talking about Qualia.

The introspective ability to hold abstract concepts in the mind is the source of the statement that humans can do so.

It is what Aristotle called the intellectual soul, not qualia. Qualia is an experience and has no relation to holding abstract concepts. As I already wrote, Turing machines cannot do it for they suffer from multiple realizability, any abstract concept can be implemented in multiple physical constructs and any physical construct can be interpreted to be any number of different abstract items. I have not written that only "wetware" can do it, I have written that turing machines cannot do it and that no purely physical construct can do it (including purely newtonian/quantum wetware).

Uh... but the neural networks (and Bayesian networks) do hold abstract concepts, and are implementable by Turing machines. So it appears that your assertion is wrong.

They do not hold concepts, they are simply physical items behaving in accord with physical principles. You as a human interpret them (wrongly) to hold concepts. You can as well interpret them to be something else. None of your interpretations change the physics and the physics have no mention of abstract concepts and can be interpreted to be any number of things. None of the interpretations make that particular interpretation a concrete implementation of it.

So if we are not purely physical constructs, what are we? Are you a dualist?

human

Are you are dualist? In the sense of Descartes' Dualism?

Banks is referencing the idea of "vitalism" - that living things have some special quality that nonliving things lack. It's kind of like a soul: it doesn't come from any particular ability of living things (like your example of projecting an abstract).

A thinking vitalist would say that, even if a computer could do the thing you're talking about, it wouldn't be "conscious" because consciousness requires this aura that only living creatures possess.

And vitalism fails because alleged vital property is undefined or tautological.

The most fun offshoot is philosophers moving qualia (perception-associations) out of the realm of neuroscience just enough every time there's an advancement.

Any example of that?