Re:A Creepy Question We'll All Have to Answer Soon
by dimmke on Tuesday August 08, 2023 @07:17AM
(#37047283)
Read the title as “A Creepy Question We’ll All Have the Answer to Soon” and was immediately extremely intrigued. Then I read the title again and lost interest in even clicking through. Amazing what difference a word can make in a title.
Re:A Creepy Question We'll All Have to Answer Soon
by ChatGTP on Tuesday August 08, 2023 @07:27AM
(#37047347)
But actually I think if the AI systems were to survive they would need to learn to cooperate and probably understand each others direction or have empathy.
Imagine 5 super intelligences that hated each other and just wanted to wipe each other out? They’d get nothing done…
If they had conflicting goals, they’d have to first stop one another from screwing up their plans…
Re:A Creepy Question We'll All Have to Answer Soon
by choudharism on Tuesday August 08, 2023 @07:28AM
(#37047357)
I like John and his body of work, but this and the general slew of posts with very little actual knowledge or expertise making grandiose / prophetic assertions are the soothsayings of our age - people convinced that they've cracked "it", but without any factual reasoning as to why.
They raise philosophical assertions disguised as logical / scientific assertions.
> “Better.”
> That is a concept that cannot be made via cold calculation. A universe boiled down to particles and numbers cannot care whether atoms arranged as a suffering plague victim are “better” than those arranged as a healthy child, or a plant, or a cloud of gas.
It can, John. The engineers teach machines what's good and bad. That's the first step in creating silicon intelligence.
> It is impossible to understand pain without having felt it yourself; it does not convey as an abstract value. I think the machine needs to tangibly interact with human suffering in the same way the Roomba’s wheels need to physically touch the floor.
Pain is nature's deterrence mechanism, bred into us through evolution. Deterrence mechanisms can take many shapes.
> This leads us to the ultimate cosmic joke: This task requires answering for this machine questions that we ourselves have never been able to answer. Let’s say I’m right and this system won’t exist until we can teach it emotion, and the capacity to suffer. How long until it asks the simple question, “What is our goal, long-term?” Like, as a civilization, what are we trying to do? What are hoping the superintelligence will assist us in achieving?
I hate being disingenuous and make a flippant argument here, but reading this from start to finish makes me ask a bad faith question - your assertion is that we won't have self driving cars because we can't tell the robots what to do with the whole civilisation right away? There's a whole spectrum of amazing (or terrible) possibilities between the extremes, my guy.
Re:A Creepy Question We'll All Have to Answer Soon
by defrost on Tuesday August 08, 2023 @07:36AM
(#37047423)
Opportunistic, and morally equivalant to the destruction of the other 72 Japanese cities, and the bombing of European cities by both the Allies and the Axis.
Re:A Creepy Question We'll All Have to Answer Soon
by choudharism on Tuesday August 08, 2023 @03:20PM
(#37054154)
I’m saying that it is possible to learn how to optimise for “better” or “worse” decisions without being trained on an opinionated dataset of every single event that has happened. An “intelligence” could exist without necessarily answering your nukes question.
You’re bringing alignment to a capability discussion.
Re:A Creepy Question We'll All Have to Answer Soon
by ChatGTP on Tuesday August 08, 2023 @09:46PM
(#37057800)
How do you suppose we do this? It makes decisions based on numbers alone ?
Personally, I don't think there is a right decision, it was just a decision which had an outcome, for some people it was ab absolutely fucking devastating decision.
This is where I think we have blind spots when developing these systems, I don't think there is a "right" or best answer here. Just some answer.
Re:A Creepy Question We'll All Have to Answer Soon
by taylodl on Tuesday August 08, 2023 @12:42PM
(#37051792)
You don't need to define good and bad, instead you focus on better and worse and the metrics used to measure them. Now the goal is to maximize "better" and minimize "worse." You may recognize this as the essentials utilitarianism. The advantage utilitarianism has is it can be applied algorithmically without passion or emotion - in other words, by AI.
Utilitarianism leads to controversial outcomes, but every decision is defensible.
Re:A Creepy Question We'll All Have to Answer Soon
by ben_w on Tuesday August 08, 2023 @02:05PM
(#37053106)
For each thing T, that T is defensible under at least one ethical framework.
Teaching an optimiser AI any of those frameworks, or even any preference ordering or combination function within Utiliarianism (because value({T, T}) doesn't have to equal 2 * value({T})), will lead to it optimising what you said, without necessarily limiting that to situations anything close to the training distribution.
To put it another way: if you run an AB test on a social media site and it observes that people are more likely to engage with content that makes them angry, then tell it to boost engagement "because socialising is always good, obviously" then it will get your users as angry as possible and suddenly you get Buddhists going off and committing surprise genocide before anyone tells you something has gone wrong.
Re:A Creepy Question We'll All Have to Answer Soon
by taylodl on Tuesday August 08, 2023 @04:55PM
(#37055321)
I would argue this has been known for decades and is in fact the origin of one of the earliest memes in computer science: To err is human, but to really mess things up requires a computer!
Re:A Creepy Question We'll All Have to Answer Soon
by jmye on Tuesday August 08, 2023 @05:18PM
(#37055602)
> It can, John. The engineers teach machines what's good and bad. That's the first step in creating silicon intelligence.
I don’t think a simple “actually, no” is a useful counter argument to his point, especially when he used multiple examples and you just shook your head.
> I hate being disingenuous and make a flippant argument here, but reading this from start to finish makes me ask a bad faith question - your assertion is that we won't have self driving cars because we can't tell the robots what to do with the whole civilisation right away?
That’s not at all his specific argument about self-driving cars (there were entire sections about the decision making issues he saw), but is the crux of his argument about AI as a whole.
> There's a whole spectrum of amazing (or terrible) possibilities between the extremes, my guy.
And there’s a whole article discussing some of those, if you read the parts in the middle. I don’t think the dude’s necessarily right, but I’m also pretty sure you missed a lot of the text in an apparent race to summarily dismiss it.
Re:A Creepy Question We'll All Have to Answer Soon
by choudharism on Wednesday August 09, 2023 @08:05AM
(#37061407)
> I don’t think a simple “actually, no” is a useful counter argument to his point, especially when he used multiple examples and you just shook your head.
I shook my head because they were similar and equally bad.
> That’s not at all his specific argument about self-driving cars (there were entire sections about the decision making issues he saw), but is the crux of his argument about AI as a whole.
I was talking about the crux of his argument, which starts by saying that we won't get self-driving cars because <start of bad arguments>.
> And there’s a whole article discussing some of those, if you read the parts in the middle. I don’t think the dude’s necessarily right, but I’m also pretty sure you missed a lot of the text in an apparent race to summarily dismiss it.
In my complete reading, the article was geared towards discussing the impossibilities (rather than possibilities) from the POV of someone who hasn't taken any time to understand the cutting edge of research that he is critiquing. That was, in fact, my original point - critique is great, but don't mask personal intuition (potentially uneducated) as science.
Re:A Creepy Question We'll All Have to Answer Soon
by ben_w on Tuesday August 08, 2023 @07:32AM
(#37047387)
An interesting thought experiment.
I aver the negative, on the grounds that, even for humans, recognising emotional states well enough to engage with other humans is independent of empathising with them. Narcissistic personality disorder[0] appears to be an example of the former without the latter, while ASD[0] appears to be an example of the latter without the former.
[0] judging by the popular descriptions, I'm not qualified in this field
Re:A Creepy Question We'll All Have to Answer Soon
by ben_w on Tuesday August 08, 2023 @08:22AM
(#37047883)
I think that premise requires the AI to have qualia, but other than that I can't really say anything; I'm not even sure what you're asking me to what-about.
Re:A Creepy Question We'll All Have to Answer Soon
by ChatGTP on Wednesday August 09, 2023 @07:16AM
(#37061030)
There definitely seems to be a lot of assumptions being made the AI won't have qualia, let's hope the assumptions remain true, but I don't imagine it will be the case forever?
Re:A Creepy Question We'll All Have to Answer Soon
by ben_w on Wednesday August 09, 2023 @08:23AM
(#37061572)
As we don't yet, so far as I know, have a testable hypotheses for the necessary and sufficient conditions for qualia, I think we can neither confirm that any given AI (or animal, including both cattle and other humans) does or does not have it.
Re:A Creepy Question We'll All Have to Answer Soon
by js8 on Tuesday August 08, 2023 @07:34AM
(#37047409)
I think whether something is painful or not depends on the perspective. Humans can voluntarily suffer lots of pain for a good cause. And vice versa, small things can be extremely painful if we don't understand them.
So it comes down to your expectations. In reinforcement learning, this is very clear - bad predictions of your future actions lead to negative reward, i.e. pain. We could almost say pain depends on your expectations of reality (however the expectations are not always consciously expressed).
So you can torture a human, because our nerves have learned (have the expectation) of not being in pain as a normal state. If they could learn more (like RL agent), they would eventually learn that the pain is just nature of the world, and ignore it.
Similarly, Data can ignore pain of his missing hand because he trusts the Enterprise crew to put it back. If they didn't and betray him, or if he hurt his arm so badly it couldn't be replaced, then I think, on some level, he would feel pain.
Re:A Creepy Question We'll All Have to Answer Soon
by trabant00 on Tuesday August 08, 2023 @07:35AM
(#37047411)
I'm having a hard time understanding these ethical questions about things that we haven't proven we can build and I'm very skeptical about the "Soon" part. It all seems like trying to build an air traffic control model in the 1800s back when we only had dreams about flying.
Even in Asimov's SF novels humanity iterates through several models by ways of trial and error over hundreds/thousands of years before arriving at somewhat stable laws.
Re:A Creepy Question We'll All Have to Answer Soon
by hug on Tuesday August 08, 2023 @07:40AM
(#37047461)
The author fails in imagination, expecting that future computer intelligence needs to follow the path of human intelligence, where it can’t be immediately snapshotted, reset, rebirths, or fed new information.
Is it cruelty to delete an AI without it knowing that might happen? Is it cruelty to inject it with feelings of euphoria for its pending deletion?
Two myopic thoughts that reach further than the article’s premise.
Re:A Creepy Question We'll All Have to Answer Soon
by botanical57 on Tuesday August 08, 2023 @08:37AM
(#37048085)
This is an interesting read and raises some questions about whether meta knowledge about suffering is sufficient to understand suffering, and more broadly, the human experience. However, it seems to leak the philosophical lean of the author and states it as a fundamental fact of the universe. This is the concept of "Good", which I believe pre-and-post Socratic thinkers wrestled with intimately. Likewise, they tended to assume Goodness came before humanity's role in it. Allow me to take the author's example and apply my own philosophical lean.
...
Robot: “So if it was just the classmate and I on a desert island, with no larger society to worry about, the act would no longer be immoral?”
Jason: “No, what you did would be wrong even if you were the only two people in the universe!”
Robot: “Why?”
Me: "It would be immoral to our - in this case, hypothetical - society because it is predicted by our values. Morality is relative, and as much as we have adapted to our physical environment, we also adapted to a social one. Morality is a device that natural selection invented to establish order and harmony within a population, and it has done a pretty good job."
Robot: "If there is no logical or fundamental reason behind it being wrong, then why do you care? Why should anyone care? Why should I care?"
Me: "The moral values of a society are internalized, by varying degree, in every member of it. Therefore, it upsets me to see these values violated, and I will expend energy - even if it disadvantages me - to amend that violation. This is how humans are built, and for most people it is far easier to live in a society when you share those internalized values. There is no reason for you to care. However, you were created with those values in mind, and you exist within a society which upholds them. In the same fundamentally meaningless way you find purpose in buying yourself a new hat, perhaps you can find purpose in honoring our values, too. For there would be no hat without us."
...
My rambling, however, has probably only proven the author right from another angle. The moral rule book is arbitrary. If the robot is cold and calculating, it has no reason to respect it whatsoever. Maybe the only way to have them care is to build them like us, whatever that would entail.
Re:A Creepy Question We'll All Have to Answer Soon
by ukuuru on Tuesday August 08, 2023 @08:39AM
(#37048114)
Human level AI is unlikely but near human level will do just like every other prodcut engineered to do its thing around us. In theory a programmable entity that feels would have infinite applications but an entity that can quantize feelings can cover lets say 90% of those applications.
No.
Thanks for coming to my TED talk
Imagine 5 super intelligences that hated each other and just wanted to wipe each other out? They’d get nothing done…
If they had conflicting goals, they’d have to first stop one another from screwing up their plans…
but that's just my biobrain thinking :(
They raise philosophical assertions disguised as logical / scientific assertions.
> “Better.”
> That is a concept that cannot be made via cold calculation. A universe boiled down to particles and numbers cannot care whether atoms arranged as a suffering plague victim are “better” than those arranged as a healthy child, or a plant, or a cloud of gas.
It can, John. The engineers teach machines what's good and bad. That's the first step in creating silicon intelligence.
> It is impossible to understand pain without having felt it yourself; it does not convey as an abstract value. I think the machine needs to tangibly interact with human suffering in the same way the Roomba’s wheels need to physically touch the floor.
Pain is nature's deterrence mechanism, bred into us through evolution. Deterrence mechanisms can take many shapes.
> This leads us to the ultimate cosmic joke: This task requires answering for this machine questions that we ourselves have never been able to answer. Let’s say I’m right and this system won’t exist until we can teach it emotion, and the capacity to suffer. How long until it asks the simple question, “What is our goal, long-term?” Like, as a civilization, what are we trying to do? What are hoping the superintelligence will assist us in achieving?
I hate being disingenuous and make a flippant argument here, but reading this from start to finish makes me ask a bad faith question - your assertion is that we won't have self driving cars because we can't tell the robots what to do with the whole civilisation right away? There's a whole spectrum of amazing (or terrible) possibilities between the extremes, my guy.
Was the use nuclear weapons at the end of WW2 good or bad ? For example …
Good v. Bad seems a tad simplistic.
Are you saying that everyone feels strong one way or another?
You’re bringing alignment to a capability discussion.
Personally, I don't think there is a right decision, it was just a decision which had an outcome, for some people it was ab absolutely fucking devastating decision.
This is where I think we have blind spots when developing these systems, I don't think there is a "right" or best answer here. Just some answer.
Utilitarianism leads to controversial outcomes, but every decision is defensible.
Teaching an optimiser AI any of those frameworks, or even any preference ordering or combination function within Utiliarianism (because value({T, T}) doesn't have to equal 2 * value({T})), will lead to it optimising what you said, without necessarily limiting that to situations anything close to the training distribution.
To put it another way: if you run an AB test on a social media site and it observes that people are more likely to engage with content that makes them angry, then tell it to boost engagement "because socialising is always good, obviously" then it will get your users as angry as possible and suddenly you get Buddhists going off and committing surprise genocide before anyone tells you something has gone wrong.
I don’t think a simple “actually, no” is a useful counter argument to his point, especially when he used multiple examples and you just shook your head.
> I hate being disingenuous and make a flippant argument here, but reading this from start to finish makes me ask a bad faith question - your assertion is that we won't have self driving cars because we can't tell the robots what to do with the whole civilisation right away?
That’s not at all his specific argument about self-driving cars (there were entire sections about the decision making issues he saw), but is the crux of his argument about AI as a whole.
> There's a whole spectrum of amazing (or terrible) possibilities between the extremes, my guy.
And there’s a whole article discussing some of those, if you read the parts in the middle. I don’t think the dude’s necessarily right, but I’m also pretty sure you missed a lot of the text in an apparent race to summarily dismiss it.
I shook my head because they were similar and equally bad.
> That’s not at all his specific argument about self-driving cars (there were entire sections about the decision making issues he saw), but is the crux of his argument about AI as a whole.
I was talking about the crux of his argument, which starts by saying that we won't get self-driving cars because <start of bad arguments>.
> And there’s a whole article discussing some of those, if you read the parts in the middle. I don’t think the dude’s necessarily right, but I’m also pretty sure you missed a lot of the text in an apparent race to summarily dismiss it.
In my complete reading, the article was geared towards discussing the impossibilities (rather than possibilities) from the POV of someone who hasn't taken any time to understand the cutting edge of research that he is critiquing. That was, in fact, my original point - critique is great, but don't mask personal intuition (potentially uneducated) as science.
I aver the negative, on the grounds that, even for humans, recognising emotional states well enough to engage with other humans is independent of empathising with them. Narcissistic personality disorder[0] appears to be an example of the former without the latter, while ASD[0] appears to be an example of the latter without the former.
[0] judging by the popular descriptions, I'm not qualified in this field
So it comes down to your expectations. In reinforcement learning, this is very clear - bad predictions of your future actions lead to negative reward, i.e. pain. We could almost say pain depends on your expectations of reality (however the expectations are not always consciously expressed).
So you can torture a human, because our nerves have learned (have the expectation) of not being in pain as a normal state. If they could learn more (like RL agent), they would eventually learn that the pain is just nature of the world, and ignore it.
Similarly, Data can ignore pain of his missing hand because he trusts the Enterprise crew to put it back. If they didn't and betray him, or if he hurt his arm so badly it couldn't be replaced, then I think, on some level, he would feel pain.
Even in Asimov's SF novels humanity iterates through several models by ways of trial and error over hundreds/thousands of years before arriving at somewhat stable laws.
Is it cruelty to delete an AI without it knowing that might happen? Is it cruelty to inject it with feelings of euphoria for its pending deletion?
Two myopic thoughts that reach further than the article’s premise.
See, machines do have emotions after all! Case closed I say!
... Robot: “So if it was just the classmate and I on a desert island, with no larger society to worry about, the act would no longer be immoral?”
Jason: “No, what you did would be wrong even if you were the only two people in the universe!”
Robot: “Why?”
Me: "It would be immoral to our - in this case, hypothetical - society because it is predicted by our values. Morality is relative, and as much as we have adapted to our physical environment, we also adapted to a social one. Morality is a device that natural selection invented to establish order and harmony within a population, and it has done a pretty good job."
Robot: "If there is no logical or fundamental reason behind it being wrong, then why do you care? Why should anyone care? Why should I care?"
Me: "The moral values of a society are internalized, by varying degree, in every member of it. Therefore, it upsets me to see these values violated, and I will expend energy - even if it disadvantages me - to amend that violation. This is how humans are built, and for most people it is far easier to live in a society when you share those internalized values. There is no reason for you to care. However, you were created with those values in mind, and you exist within a society which upholds them. In the same fundamentally meaningless way you find purpose in buying yourself a new hat, perhaps you can find purpose in honoring our values, too. For there would be no hat without us." ...
My rambling, however, has probably only proven the author right from another angle. The moral rule book is arbitrary. If the robot is cold and calculating, it has no reason to respect it whatsoever. Maybe the only way to have them care is to build them like us, whatever that would entail.