---

IIT is the most coherent definition of consciousness I'm aware of https://en.m.wikipedia.org/wiki/Integrated_information_theor...

People dismiss it often, and it has valid academic criticisms, but most "lay" critiques I've seen seem to dismiss it because it gives things they don't consider to be "conscious like them" consciousness.

I think in general, we overexagerate qualities that humans have as uniquely special and important (intelligence, sentience, consciousness), even if the definition of them is fuzzy and any non-fuzzy definition is "too inclusive". I wonder if this is because we are collectively Identity building and creating an "other" out of nature, because otherwise we'd

1. Be much less special as we want to fee

2. Would have to consider a lot of inconvenient externalities for our reasoning (moral and otherwise) to be consistent

And both are slowdowns when you are bootstrapping a post scarcity society out of scarcity, so it'd culturally valuable to reify special qualities that we just determine "we" possess and "they" don't - because it's easier to unify on and allows more actions than making sure everyone can deal with the unfiltered reality of human... unremarkableness in an uncaring world, another social species like so many other with a temporary oligarchy on the planet (without wanting to drone to philosophically, I do think this aspect of Weltschmerz is underappreciated, especially seeing the anxiety amongst my peer group when the topic comes up)

If your attempt at a formalism provides useful, actionable implications, and only really disagrees with people's intuitions on cases that are generally believed to be edge cases, or maybe that you also have good reason to believe are edge cases on different ground, then you might have a decent definition.

If your attempt at a formalism provides useful implications, but disagrees with everyone's intuitions on what the words you're using mean, you've probably defined the wrong thing. That's not to say that you aren't defining something sensible (you might still be), but strongly suggests it's not what you want it to be.

Selling IIT as a description of consciousness is roughly the same flavor of hubris as defining berries to include oranges and cucumbers and exclude raspberries and raspberries.

Not trying to defend IIT, but the only reason your example works here is because from our human perspective (eating these plants) we find it absurd to include cucumbers and exclude raspberries.

So talking about hubris: Wouldn't plants (if they could) very likely have a totally different perspective? To then it would probably seem totally absurd that we use taste or shape of the fruit as a grouping feature and not something that might be totally more defining that plants can experience.

Of course it is intuitively understandable to look at ourselves and animals we understand when defining consciousness. Even like that it is hard to define. Trying to incorporate things that show conscious-like behaviors, but in totally human-unrelatable ways is not a bad next step, but it is hard to know where to draw the border. A rock could have conscious processes running in it on geological timescales — how would one falsify that?

Note that the issue that you are concerned with here and the usefulness of IIT are separate concerns, and critics like Aaronson are not taking that position on the basis of the attitudes you claim are behind many 'lay' critiques.

But even accepting that IIT is wrong in the "consciousness might imply high IIT score, but not vice versa" sense, I'm simply not aware of any definition which is as *attackble* because of it's formal specification.

This is not me going "well you don't have anything better", but me asking to be pointed to similarly strictly defined notions of consciousness

In the same ballpark as IIT, I've seen many mentions of GWT/GNT (global neuronal workspace theory). But the PDF covers that and many others. I don't have any particular "conclusion" myself, but it's just a fascinating discussion to be swimming in even when utterly clueless.

EDIT: And I forgot the most obvious link that I actually had read in the past, Stanford's page on conciousness talks about theories: https://plato.stanford.edu/entries/consciousness/#SpeTheCon

If you add that to IIT, then you suddenly get a new filter for what could be conscious - something that competes for information replication, not just something with high IIT score.

Is that not just your intuition too? Is it not possible to imagine an artificial consciousness that does not care one way or the other for its own survival?

But you can piggyback AI training on human culture and embody the agent in our environment so it can still function like a conscious biological agent. The AI is just passing the buck to humans, without us it wouldn't even have a goal.

On the other hand all living things have goals without human help. Maybe when AIs can self replicate their information into the future like us, without external help, they could be eventually conscious. We don't know yet how to make physical self replicators that evolve. Only nature solved this riddle so far.

BTW, AlphaGo was subjected to evolutionary techniques, and it shows. The agents had to duel with clones and past versions of themselves. Survival was conditioned on win rate. Suddenly AI can surpass humans in this little environment, but that came at the price of discarding so many unfit earlier versions.

That is his entire pitch; if you've not read it I wholly recommend it.

That's probably because the rest of researchers ignore it like the plague. If everybody had to comment on it for some magical reason, it would be probably be burned down to the ground and then some.

https://en.m.wikipedia.org/wiki/Mind–body_dualism

Also interesting thought experiment is the philosophical zombie, lots to read by David Chalmers, etc.

No, you think that's possible but it's naive. Your existence depends on environment and self replication of information. Your structure depends on it, your cognition. Are you saying you can be separate from the tree, and everything just a fake around you?

In other words if we can observe a machine thinking about its own (symbolic) thinking, then I think it's conscious.

Consciousness is the ability to observe one's own mental processes and make sense of them symbolically. That is one possible definition of it.

You are right that we can't really prove that humans are conscious, because we cannot observe their thinking.

But, to prove that a machine is conscious all we need is to agree on a definition of what it means to be conscious, because machines can be easily inspected. And then find a machine that demonstrates features that fit the definition.

If we can't agree on what it means for something to be conscious then the whole question of whether some AI is or could be conscious is of course meaningless.

I have the feeling that attempts to create conscientious AI will just result in paperclip maximizers that have to use rationalization as an indirection layer to engage in their paperclip maximizing behavior.

Not much of an improvement, but realistically, what other result would be tolerated by whoever is building a DWIM AI to run a paperclip factory and for whom conscientiousness would just be a regulatory box to tick?

I'd be very skeptical of claims that, say, some complex ongoing control system, like a self-driving car, could be "conscious". But there's an argument someone could make.

But an artifact that has no data storage seems to fail any reasonable definition immediately. Maybe it could be part of something else that you can claim is conscious if you add storage, output controls, aims or whatever. But by itself the claim just seems preposterous.

Furthermore, one can argue that consciousness is a requirement for knowledge.

Perhaps that's way off, I was just starting to think along those lines as I came to your comment, and it seemed it might fit, that we might be thinking along the same lines.

I'll give it to cats and dogs, birds, the apes, and many other critters, but not AI. Those things are separate developmental tracks, and not included in AI research, from what I can see. They aren't going to spontaneously arise, and it seems to me that they all came before we achieved sentience at all.