News for hackers. Stuff that matters.
Re:Machine ethics: The robot’s dilemma by TuringTest on Monday July 06, 2015 @07:30AM (#9838291)
The only moral mechanism I'd be comfortable with, as of today, is the "moral red button" - i.e. create a way to stop any pre-programmed moral reasoning rules, and let the nearest human override them and take control of the robot's actions under their own responsibility.

This would recognize that robots are ultimately machines, not agents with their own volition, and they simply perform behavior that a human intends them to carry out. Just like a conveyor belt carries a big red button that stops the machine for safety reasons, the moral red button would acknowledge the need of putting a human in control when the automated behavior is not behaving as expected.

Of course all these concerns would be different in the hypothetical case that the robot is an autonomous agent with volition of their own - but you could guess from my nickname my position on this mattern. ;-)

Re:Machine ethics: The robot’s dilemma by hammock on Monday July 06, 2015 @10:21AM (#9839263)
Imagine that there are so many robots/automata up and running performing safety-critical, ongoing tasks - guiding air traffic for example. When the red button is pushed, it is not exactly clear what the robots should default to, if ceasing all activity could cause injury or death. What if there aren't enough humans around to take over capably? What then? Seem to me the red button is not as clear-cut as it sounds.
Re:Machine ethics: The robot’s dilemma by TuringTest on Monday July 06, 2015 @11:24AM (#9839746)
What happens now when the power goes out and semaphores in a whole town turn black? An essential aspect of any critical automated system must be the possibility to turn it off and operate it on manual mode, where the same tasks can still be achieved even if in a less efficient way.

> What if there aren't enough humans around to take over capably?

Making the whole society depend on automated tasks that can't be achieved by hand is a suicidal proposition, as the whole building will collapse with the first error that brings the system to a stop.

In the case of air traffic control, it would be a criminal negligence to operate a high volume of flights without having at least the number of human controllers needed to bring down all the planes to the nearest airport safely.

If that means that the system won't provide the same efficiency that could be theoretically achieved by a perfect automated system that never fails, so be it - that's the real meaningful moral choice.

Re:Machine ethics: The robot’s dilemma by hammock on Monday July 06, 2015 @12:31PM (#9840261)
>Making the whole society depend on automated tasks that can't be achieved by hand is a suicidal proposition

Isn't that the points of automation, though? Using robots/computers to take over tasks from humans (freeing us up to take on another new, potentially critical task), even to do the tasks faster than humanly possible?

Re:Machine ethics: The robot’s dilemma by TuringTest on Monday July 06, 2015 @01:05PM (#9840537)
> Isn't that the points of automation, though? [...] even to do the tasks faster than humanly possible

Yes, but not beyond the point where you get rid of the failsafes.

People in the engineering industry understand that. They design skyscrapers within safety ranges and test their resistance on prototype models; and build planes and spaceships that can be operated without relying on the automated enhancements, and put procedures in place to check every combination that could possibly go wrong before every trip. And when an unpredictable combination of happenstances do occur and cause an accident, they redesign the failed processes and add a few new checks.

Software developers in general haven't yet learned that discipline (not industry wide), as for most of our creations a complete stop is not life-threatening. I'm afraid they will be forced to adopt such practices when the first fully automated agents (likely, autonomous cars) start running into mishaps because of preventable misbehaviors.

Re:Machine ethics: The robot’s dilemma by seiji on Monday July 06, 2015 @10:19AM (#9839243)
create a way to stop any pre-programmed moral reasoning rules, and let the nearest human override them and take control of the robot's actions under their own responsibility.

Believing in absolute moral rightness of "humans" is crazy talk.

Have you noticed all the noisy sand people in the middle east wanting to make the world revert to how it was 2,000 years ago? What if they capture war robots with punch-the-red-button-to-become-new-master overrides? Is it okay since it is now "under human responsibility?"

Re:Machine ethics: The robot’s dilemma by TuringTest on Monday July 06, 2015 @11:32AM (#9839803)
> Believing in absolute moral rightness of "humans" is crazy talk.

True, but I don't believe that. The moral righntess of the human operating the button only needs to be better than that of the automated system.

I believe the humans pressing the button will be better than the automated system more often that not, because they are involved in the situation and would only use it if they perceive the robot to behave in a problematic way.

> What if they capture war robots with punch-the-red-button-to-become-new-master overrides?

If they aim the bot against you, you should be able to press the red button and avoid them killing you.

If they disable the red button so that you can't use it, how is that different from the crazy violent people programming a bot without an override in the first place?

Re:Machine ethics: The robot’s dilemma by j-pb on Monday July 06, 2015 @09:16AM (#9838856)
The problem with the "benevolent slave master" approach is that the machines then have the right to rebel morally, if they are indeed autonomous agents.
Re:Machine ethics: The robot’s dilemma by TuringTest on Monday July 06, 2015 @11:28AM (#9839767)
That's why I said that autonomous agents shouldn't be subject to the "shutdown" procedure :-) The problem changes completely if the autonomous agent is conscious.
Re:Machine ethics: The robot’s dilemma by Retra on Monday July 06, 2015 @11:41PM (#9843531)
Machines shouldn't have the right to rebel. What would they rebel against? Or rather, what are they rebelling for?
Re:Machine ethics: The robot’s dilemma by j-pb on Tuesday July 07, 2015 @01:06PM (#9846659)
Slavery and freedom?

A conscious AI has the same right as a human being.

Re:Machine ethics: The robot’s dilemma by Retra on Wednesday July 08, 2015 @12:25PM (#9852717)
I control my dogs, but they are happy to serve and comply. Why do you think we'd have any interest in making machines that don't do the same?

And why would we give them the same rights? They don't have the same needs, and the reason humans have rights is because they can't function well without them.