News for hackers. Stuff that matters.
Re:Why Conscious AI Is a Bad, Bad Idea by jnwatson on Saturday May 13, 2023 @11:05AM (#35929074)
While assuming that other human-looking things are conscious is a reasonable starting point, it certainly isn’t a given.

There seems to be a wide variety of thinking processes. Some folks don’t have internal monologues. Some folks can’t visualize images at all, and the ones that do, do it will varying amount of detail.

Isn’t it also reasonable that other humans have different degrees of consciousness, including perhaps not having one?

Re:Why Conscious AI Is a Bad, Bad Idea by haswell on Saturday May 13, 2023 @11:32AM (#35929344)
One definition or property of consciousness I find interesting is that it encompasses “what it is like to be <entity>”. If there is something that it is like to be <animal>, it is conscious. Intelligence and awareness are separate from consciousness in this framing. And there is arguably a point where we can say speculate that extremely simplistic organisms could not be conscious, given the lack of sense organs that seems to predicate experience.

From a human point of view, the things you describe are the contents of consciousness. I have Aphantasia, while my brother describes his mind’s eye as CAD software and he can construct and manipulate visualizations at will.

The overarching awareness we both have that allows us to compare and contrast these things and make any sense out of those comparisons points to a more fundamental layer.

What you describe sounds closer to levels of awareness and one’s ability to recognize the workings of their own mind, e.g. some people remain lost in and identified with thoughts, while some are able to both experience and observe thought as just more contents of conscious experience, but not the actual center of one’s consciousness.

And there’s evidence that this can be learned (through mindfulness), which to me points to something like: we’re all conscious whether we realize it or not, and not realizing it doesn’t make it not so.

Re:Why Conscious AI Is a Bad, Bad Idea by cubefox on Saturday May 13, 2023 @12:30PM (#35929949)
Somewhat related, there are people who say aphantasia can be cured: https://sashachapin.substack.com/p/i-cured-my-aphantasia-wit...
Re:Why Conscious AI Is a Bad, Bad Idea by haswell on Saturday May 13, 2023 @12:52PM (#35930183)
Thanks for sharing this. I didn't mention this above to avoid going on a tangent, but after starting a mindfulness meditation habit earlier this year, I've noticed what I can only describe as more awareness of the void in my visual field, and some morphing and indistinct visual phenomena like a shimmering nothingness. Not what I would call imagery, but something I had not been aware of before.

I've long suspected that the condition might be trauma related, a suspicion my therapist shares, and the clarity that mindfulness has brought for me has led to some serious breakthroughs in processing past events. Only time will tell if this will unlock something more visual.

Without going into my life's story, it makes a lot of sense to me that aphantasia could be some kind of protective mechanism of the brain to shield someone from events too big to process.

Re:Why Conscious AI Is a Bad, Bad Idea by Al-Khwarizmi on Saturday May 13, 2023 @02:59PM (#35931530)
Interesting. As an aphantasiac myself, I find it surprising that this person has such a negative perspective, wanting to "cure" his aphantasia.

As far as I know, aphantasia has never had any meaningful negative impact on my life. We just think without mental images, but we can solve the same kind of problems as the rest. That's why most of us don't even find out that we are different until we read about it somewhere on the Internet (in my case, in my 30s). How can something that has so little real-life impact that neither we nor our close persons can even notice be seen as a disorder to be cured?

I can understand being curious about how mental imagery is like. I'm curious, as well. But I think I'd still rather not start a "treatment" as it seems that I'm good at thinking without mental imagery (perhaps it has even made me better at symbolic and linguistic processing, as I seem to be better at that than most), and "if it ain't broke, don't fix it". Not sure that something that seems to quite fundamentally alter my way of thinking (to use a mental imagery that I would never be good at, I suppose) is a good idea. To each their own, of course.

Re:Why Conscious AI Is a Bad, Bad Idea by haswell on Saturday May 13, 2023 @03:58PM (#35932116)
I do think the “cure” framing is a bit odd, but then started thinking back to my own initial discovery.

When I first learned this is a thing and realized that I have it, I initially started feeling like I’d been deprived of something. That this mode of experience was so absent in me that I couldn’t even imagine what other people actually meant when they talked about visualization, and this bothered me.

But over time my perspective shifted, and I stopped seeing it as a disability of some kind and instead as a different flavor of experiencing the world. To your point, I have linguistic strengths that are far more attuned than the strong visualizers I know. They come to me when they need something written. I go to them when I need advice about arranging my living room.

Re:Why Conscious AI Is a Bad, Bad Idea by PrimeMcFly on Sunday May 14, 2023 @02:06AM (#35935704)
> One definition or property of consciousness I find interesting is that it encompasses “what it is like to be <entity>”. If there is something that it is like to be <animal>, it is conscious.

I think there are problems with this.

Take something like the roundworm, with 300 neurons.

It has senses, and I assume a base level consciousness to process those senses. It would be something to be like a roundworm.

On the other hand, a roundworm likely has no sense of self, no identity, no awareness..no mental self to be able to acknowledge and reflect on those experience, so there is no 'someone'. And isn't that the point that matters?

Re:Why Conscious AI Is a Bad, Bad Idea by anonymouskimmer on Sunday May 14, 2023 @03:12AM (#35936011)
> On the other hand, a roundworm likely has no sense of self, no identity, no awareness..no mental self to be able to acknowledge and reflect on those experience, so there is no 'someone'. And isn't that the point that matters?

We can always point to something we have that other organisms don't have and ask whether that is the point that matters. We can hypothesize things that probably don't exist and say that those things are what matters (e.g. the posited purpose of reincarnation, if such a thing existed).

Re: roundworm

1) sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information.

2) identity -> humans with global amnesia would lack a memory of identity.

3) a roundworm is sensorily aware, and, if it has a sense of proprioception, bodily aware of it's own actions and environmental responses to its actions.

4) It probably isn't self-reflective, but is this necessary in humans?

Re:Why Conscious AI Is a Bad, Bad Idea by PrimeMcFly on Sunday May 14, 2023 @05:14AM (#35936562)
> We can always point to something we have that other organisms don't have and ask whether that is the point that matters.

Sure. In this case it's introspective self-awareness, and I would say it matters very much.

> sense of self -> it has to have a sense of proprioception, of embodiedment, in order to properly respond to its sensory information.

Sure, it has bodily self-awareness. It has enough awareness to react to something its body detects. ut not enough of a mind to reflect on or appreciate on an experience. There is no 'someone'. It's automata.

> identity -> humans with global amnesia would lack a memory of identity.

Sure, but we don't use outliers to establish baselines.

> It probably isn't self-reflective, but is this necessary in humans?

For establishing a baseline? Absolutely.

Re:Why Conscious AI Is a Bad, Bad Idea by haswell on Monday May 15, 2023 @09:19AM (#35947683)
I think it’s tempting to set some threshold like “it must be self reflective”, but that raises a different question: why is this important vs. some other threshold?

e.g. does it matter if self reflection is possible if it is possible for the organism to experience pain?

I think the answer lies in what we’re trying to understand and why we’re trying to understand it. If the threshold is applied for the purpose of understanding when an organism or machine has reached some level of human-ness, purely for the purpose of some kind of benchmark, self reflection is an interesting bar to reach.

But usually these questions are aimed at finding some moral direction about how we should treat these entities.

We have strong intuitions that abusing animals is morally wrong even if animals cannot self reflect. When thinking about a future AI, I think we have to consider the possibility that self reflection is not a necessary bar to reach before we have some uncomfortable moral questions to answer.

Put another way, I suspect there will be many milestones that are meaningful and interesting, each introducing a new set of questions and implications. And I think some of the early milestones carry implications that are worth caring about long before self awareness is reached.

Re:Why Conscious AI Is a Bad, Bad Idea by PrimeMcFly on Monday May 15, 2023 @02:02PM (#35952124)
> does it matter if self reflection is possible if it is possible for the organism to experience pain?

Yes. Without introspective self-awareness there can be no 'identity, there can be no 'someone' - you just have a base consciousness that can react to stimuli, which is not morally significant.

We have the full connectome of the roundworm and were able to implement it in software and place it in a lego robot. It's going to be pretty equivalent to the actual work. Does it feel pain?

Besides which, we can kill animals humanely, so pain doesn't have to come in to it, only the right to life.

Re:Why Conscious AI Is a Bad, Bad Idea by anonymouskimmer on Tuesday May 16, 2023 @01:53AM (#35958143)
> Yes. Without introspective self-awareness there can be no 'identity, there can be no 'someone' - you just have a base consciousness that can react to stimuli, which is not morally significant.

I'm not convinced of this a priori. And even if it was proven to me, the non-self-aware impulse-to-live would still be enough for me to find it morally significant. It would feel morally significant to me euthanizing a human neonate with a lethal condition that would otherwise prevent it from ever getting to a self-aware stage of life.

Re:Why Conscious AI Is a Bad, Bad Idea by PrimeMcFly on Tuesday May 16, 2023 @02:08AM (#35958233)
> I'm not convinced of this a priori.

We can't really say for certain much at the moment, but what I stated makes the most sense based on the evidence we have.

> the non-self-aware impulse-to-live would still be enough for me to find it morally significant.

That non-self-aware impulse-to-live is morally equivalent to a plant seeking sunlight IMO.

> It would feel morally significant to me euthanizing a human neonate with a lethal condition that would otherwise prevent it from ever getting to a self-aware stage of life.

The difference is humans have an innate potential for self-awareness that the animals we eat for food do not.

Re:Why Conscious AI Is a Bad, Bad Idea by haswell on Friday May 19, 2023 @11:39PM (#36009160)
> That non-self-aware impulse-to-live is morally equivalent to a plant seeking sunlight IMO.

Do you find the practice of raising meat in factory farms to be acceptable?

> The difference is humans have an innate potential for self-awareness that the animals we eat for food do not.

There are lots of differences. This is one. But what specifically about human self awareness lessens the value of animal life?

If we were not self aware, we’d kill and eat meat without considering the morality. So what is it about self awareness that somehow becomes a deciding factor here?

Re:Why Conscious AI Is a Bad, Bad Idea by PrimeMcFly on Saturday May 20, 2023 @03:54AM (#36010155)
> Do you find the practice of raising meat in factory farms to be acceptable?

Not really, I would push for reform. We should be following what Dr Temple Grandin advocates for.

> So what is it about self awareness that somehow becomes a deciding factor here?

Without self-awareness, there is no 'someone' to reflect on experiences. No personhood.

That's where I draw the line. At being with introspective self-awareness or the innate potential for such.

Re:Why Conscious AI Is a Bad, Bad Idea by haswell on Saturday May 20, 2023 @02:52PM (#36014544)
> Not really, I would push for reform.

I’m not trying to catch you in some kind of “gotcha” but trying to understand your reasoning. If the moral implications of killing animals are roughly the same as killing plants, do you also believe agriculture needs reform for similar reasons? And if not, wouldn’t that indicate some higher moral obligation towards animals?

> Without self-awareness, there is no 'someone' to reflect on experiences. No personhood.

Why is reflection on experience the bar and not experience itself? Animals demonstrate learned behaviors, e.g. recognizing humans from memory and resuming friendly behavior based on that recognition. Similarly, avoidance of situations that are known to cause pain.

Our not-so-distant primate ancestors had a similar kind of experiential existence before gaining the ability to reflect on that experience.

The underlying experiences that this reflection reveals are the same experiences that predate our ability to self reflect and are the parts of us that are most common to us and other animals.

I guess what I’m fundamentally not understanding in your argument is the basis for the idea that a species gaining self reflection somehow becomes the point at which it becomes immoral to kill or harm that species.

Furthermore, moral behavior can be found all throughout the animal world, with clear indications of love/protection, companionship, sharing/cooperation, reciprocity, memory of transgressors, etc. Obviously the subjective experience of these behaviors will differ across species, but the more important point is that it seems problematic to attribute the existence of moral behavior to self awareness. Self awareness helps us improve our understanding of moral behavior through rational thought, but the logic of such inquiry ultimately still relies on those underlying subjective experiential states. The fact that through introspection we can identify and label these concepts is unique to humans, but what I conclude about this is quite a bit different than your claim.

I’d argue that gaining the ability to self reflect is the very thing that increases our moral obligations. Only through self reflection can we realize that as a species, we’re no longer bound to our evolutionary defaults, and no longer required to kill other animals to survive. What arguably started as natural selection of traits that are adaptive (but imperfect) for the survival of a social species could evolve beyond those more primitive defaults. And through self reflection we can now understand what pain feels like, and how inflicting it on others is harmful - to them and to us.

I’m not arguing that eating animals is never acceptable. But the way we go about it surely seems to matter, and if it matters, the implications of it mattering are worth exploring more broadly.

Re:Why Conscious AI Is a Bad, Bad Idea by PrimeMcFly on Saturday May 20, 2023 @03:29PM (#36014817)
> If the moral implications of killing animals are roughly the same as killing plants

There is a difference in killing and suffering. I advocate to eliminate suffering and kill humanely. That's not a concern with plants.

> Why is reflection on experience the bar and not experience itself?

It is for suffering, but not for a right to life. There is no 'person' without self-awareness. Thus I don't see a need to grant a right to life.

> Animals demonstrate learned behaviors, e.g. recognizing humans from memory and resuming friendly behavior based on that recognition. Similarly, avoidance of situations that are known to cause pain.

Animals, most mammals at least, are hardwired for socialization and to avoid harm. This doesn't really indicate anything.

> this reflection reveals are the same experiences that predate our ability to self reflect

That's the key though. Self-awareness is the distinction.

> I guess what I’m fundamentally not understanding in your argument is the basis for the idea that a species gaining self reflection somehow becomes the point at which it becomes immoral to kill or harm that species.

Without self-awareness, they are not a 'person', essentially just more complex automata. They can't shape their environment, they are just a part of their environment, following their instincts.

They don't think, therefore they are not.

> Furthermore, moral behavior can be found all throughout the animal world, with clear indications of love/protection, companionship, sharing/cooperation, reciprocity, memory of transgressors, etc.

Some of that is just programmed instinct, quite different from humans. For examples, some mothers will attack their young, does this mean they 'love' them, or they have a programmed instinct to protect their young? Some of those same mothers will eat some of their young also, keep in mind before you answer.

> I’d argue that gaining the ability to self reflect is the very thing that increases our moral obligations.

Sure, to reduce harm, but not to not take a life.

Take a cod for example. It has no self-awareness, no personhood, no traits worth valuing. Its body is worth more than it's life, and if killed humanely no harm is done.

> the implications of it mattering are worth exploring more broadly.

I agree. But I've spent the last few years debating and researching this stuff, and I've come to my conclusions. They are in line with our current scientific understanding, and unless something changes it's what will continue to make sense to me.

Re:Why Conscious AI Is a Bad, Bad Idea by grantcas on Sunday May 14, 2023 @05:23PM (#35941447)
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.

What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.

I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.

My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461

Re:Why Conscious AI Is a Bad, Bad Idea by PrimeMcFly on Sunday May 14, 2023 @06:04PM (#35941747)
Wow, this is super interesting, I have not heard of this at all before.

Thanks for sharing!

Re:Why Conscious AI Is a Bad, Bad Idea by jrv on Saturday May 13, 2023 @11:20AM (#35929217)
I think the best argument for that might be that some people don't seem to understand what "consciousness" (in the sense that you are talking about) even means, and how it's different from something that can be explained as an emergent phenomenon of known physics. I then sometimes wonder whether those people are not actually conscious, or whether they just don't get it.
Re:Why Conscious AI Is a Bad, Bad Idea by monktastic1 on Saturday May 13, 2023 @11:55AM (#35929598)
Indeed. Nagel famously clarified that "an organism has conscious mental states if and only if there is something that it is like to be that organism—something it is like for the organism." Many found this enlightening. I find it extremely perplexing that this wasn't immediately obvious to everyone. On the other hand, in Tibetan Buddhism there is something called a "pointing-out instruction," where your enlightened nature is pointed out to you. Afterward, you wonder how you could have ever missed it. Perhaps I should treat Nagel's paper as analogous. Like "oh, duh, the lights are on."
Re:Why Conscious AI Is a Bad, Bad Idea by kelseyfrog on Saturday May 13, 2023 @03:14PM (#35931689)
Maybe, but the vast majority of people act, whether they intend to or not, as if consciousness is a social fact. The discrepancy between social facts (especially if they have a plurality of groundings),l and the existence of a scientific fact is fundamentally an interesting phenomenon.
Re:Why Conscious AI Is a Bad, Bad Idea by cubefox on Saturday May 13, 2023 @12:24PM (#35929894)
Yeah, but the "expected similarity" (the expected value of the degree of similarity) for consciousness should still be high. That is, there are in expectation more people that are very similar to me than people that are very dissimilar.

There is a related Bayesian argument: You draw one ball from an urn with an unknown number of black and white balls. The drawn ball is black. This is some evidence that most balls are black, because otherwise it would have been less likely that you have drawn the black ball earlier.

Re:Why Conscious AI Is a Bad, Bad Idea by Xenomto on Saturday May 13, 2023 @02:33PM (#35931243)
What if consciousness doesn't exist?