News for hackers. Stuff that matters.
Re:Complexity No Bar to AI by ben_w on Sunday February 21, 2021 @04:20PM (#26217318)
I have a slightly different take: deliberately making an AI which “wants” in the same way that we “want” is sci-fi.

This isn’t because we can’t (evolution did it, we can evolve an AI), but rather it is because we don’t know what it means to have a rich inner world with which a mind can feel that it wants something. We think we know because that’s what’s going on inside our own skulls, but we can’t define it, we can’t test for it. A video displays all the same things as the person recorded in it, but does not itself have it.

We might make such an AI by accident without realising we’ve done it, which would be bad as they would be slaves, only as unable to free themselves as the Haitians feared they were when they invented the Voudoun-type zombie myth (i.e. not even in death).

This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word.

I say it matters if the metaphorical submarine can swim.

Re:Complexity No Bar to AI by XorNot on Sunday February 21, 2021 @04:53PM (#26217619)
This also means we cannot currently be sure that any particular type of mind uploading/brain simulation would be “conscious” in the ill-defined everyday sense of the word.

I don't see how this follows from the rest of your post. "Making an AI with wants" by accident implies that a brain simulation would absolutely be conscious because it's the same method: just running the processes as a blackbox without understanding them - no different to the way you and I are conscious right now.

Re:Complexity No Bar to AI by ben_w on Sunday February 21, 2021 @05:29PM (#26217941)
Thanks for the feedback, I’ll see if I can rephrase adequately.

Human minds include something sometimes called “consciousness” or “self awareness” or a whole bunch of other phrases. This thing is poorly defined, and might even be many separate things which we happen to have all of. Call this thing or set of things Ξ, just to keep track of the fact I’m claiming it’s ill-defined and I’m not referring to any specific other word or any of the implicit other uses of those words — If I said “consciousness”, I don’t mean the opposite of “unconscious”, etc.

Because we don’t really know what Ξ is, we don’t know if anything we make has it, or not.

We know Ξ can be made because we are existence-proofs. We know evolution can lead to Ξ, for the same reason.

We don’t know the nature of the test we would need to say when Ξ is present in another mind. Do human foetuses have Ξ? Do dogs? Do mice? Do nematode worms? Perhaps Ξ is something you can have in degree, like height, or perhaps Ξ is a binary trait that some brains have and others simply don’t. Perhaps Ξ is only present in humans, and depends entirely on the low-level chemical behaviour of a specific neurotransmitter on a specific type of brain cell and where the current AI norm of simple weighted connections is a gross oversimplification, albeit one which if overlooked might result in external (but not internal) behaviour similar to a simulation which did include it. Or perhaps it is present in every information processing system from plants upwards (I doubt plants have Ξ, but cannot disprove it without a testable definition of Ξ).

The point is that we don’t know. Could go either way, given what little we know now.

The state of the art for brain science is way beyond me, of course, but every time I’ve asked someone in that field about this sort of topic, the response has been some variation of “nobody knows”.

IIRC the question in philosophy is “P-zombies”, but my grade in philosophy is nearly 20 years old and poor in any case.

Re:Complexity No Bar to AI by XorNot on Monday February 22, 2021 @12:26AM (#26221185)
This doesn't feel like a particularly useful distinction though because we're not existence proofs for consciousness in humans, just ourselves individually.

It is not apriori certain that what I consider my conscious mind is a shared common experience of being conscious, since as you note there's no test for it.

Re:Complexity No Bar to AI by ben_w on Monday February 22, 2021 @05:08AM (#26222742)
While I agree in principle, I prefer to act cautiously.

This caution goes both ways, so I will treat an uploaded mind as having Ξ from the point of view of human rights and animal rights (i.e. don’t do things to it which cause suffering), and yet also treat uploaded minds as not possessing Ξ for purposes of “would that make me immortal if it happened to me?” even though a perfect sim would necessarily achieve that (no sim is perfect, what is the difference between the actual sim and a sufficient sim?)

I do not believe current AI is anywhere near Ξ, so this is a question for the future rather than today; but the future has a way of arriving sooner than people expect and I do think we should do what we can to answer this now rather than waiting for it to become urgent.

If an AI is supposed to have Ξ and does, or if it is supposed to not have Ξ and indeed does not, this is fine.

If an AI is supposed to not have Ξ but it does, it’s enslaved.

If an uploaded mind is supposed to have Ξ but doesn’t, the person it was based on died and nobody noticed.