Machine ethics: The robot’s dilemma

http://www.nature.com/news/machine-ethics-the-robot-s-dilemma-1.17881

by r3bl • 11 years ago

33 30 11 years ago

BetaCygni2• 11 years ago

They reference Asimov. Good. His rules should provide enough inspiration and he explored them thoroughly.

For example: the zeroth law (preserve humanity) might help in deciding between a child and an adult. If the adult is beyond reproductive age (e.g. elderly) it may save the child. If the adult is still able to reproduce and the child would not survive on its own, save the adult. In case of doubt it should probably use quality adjusted life years, and that will probably cause it to save the child.

TuringTest8• 11 years ago

I would be terrified by a machine that took those kind of decisions over human life without taking into consideration the specifics of each particular situation, merely from an abstract rule that some engineers thought of and derived from first principles in their laboratory, without input from the real world or from a limited sample of cases. In your hypothetical example, what if the elder was the only leader of a faction that could negotiate a tready and avoid a global war, but the robot has no built-in rules to detect such development and kills the elder anyway?

There's a reason why we keep judges around to interpret and implement the law, even if the rules are established by an independent body and are in theory are designed to maximize the common good. The moral of Asimov stories was that it is impossible to encode morality as a simple set of rules, as the complexity of real life will always produce unpredictable situations, and ethics will provide ambiguous answers depending on which principles are considered more important.

1stop• 11 years ago

But doesn't this miss the fact that "humanity" is emotionally driven and making decisions based on pure logic is un-human?

We also have a brain with multiple subsystems that can override each other, and we see this behaviour as "normal" (i.e we don't get confused and puzzled by a fat person eating, thinking to ourselves "I don't get it, they clearly don't need the food... ?" - Where as if someone stopped eating mid meal and said "I've reached sufficient caloric intake" we would be confused and puzzled).

seiji2• 11 years ago

Where as if someone stopped eating mid meal and said "I've reached sufficient caloric intake" we would be confused and puzzled.

You clearly don't live in San Francisco. Talking like an asocial robot is encouraged to be part of the in-crowd.

TuringTest8• 11 years ago

making decisions based on pure logic is un-human?

It's not just un-human, it's impossible. Every logic decision ultimately depends on the axioms deciding what is a "good" and a "bad" decision, and the choice of axioms needs to be based on an emotion - as choosing one set of axioms over another is otherwise arbitrary.

noreasonw• 11 years ago

Just suppose the adult say to the robot: I will give you this big check, with that money you can save the life of many thousand people that are starving from malnutrition, would the robot accept the check?

simonh3• 11 years ago

Have you even read I Robot? You now, the book in which the laws of robotics lead the robots to institute a robot ruled dictatorship, taking away human freedoms because they reasoned by by controlling our lives completely they could protect us from ourselves? The whole point of Asimov's laws was that naively encoded moral rules can have horrific consequences.

TeMPOraL• 11 years ago

Exactly. It's surprising how Asimov's Laws became recognized in the popular culture as a well-thought-out solution to the morality issues of AIs, when their whole point was to be an example of how to not do things.

eli_gottlieb2• 11 years ago

That was the movie version of I, Robot. In the book version, people handed control over to the Machines because they found that the Machines could plan economic and governmental matters better than people could. Also, in Asimov's literary canon, the robots ultimately destroy themselves on the reasoning that having them around was crippling humanity.

BetaCygni2• 11 years ago

I've read quite a bit of Asimov, did you notice it's the same story over and over again?

I, for one, welcome our new robotic overlords.

Retra3• 11 years ago

I doubt a machine making such simple evaluations would be able to make those kinds of decisions at all.

TuringTest8• 11 years ago

The only moral mechanism I'd be comfortable with, as of today, is the "moral red button" - i.e. create a way to stop any pre-programmed moral reasoning rules, and let the nearest human override them and take control of the robot's actions under their own responsibility.

This would recognize that robots are ultimately machines, not agents with their own volition, and they simply perform behavior that a human intends them to carry out. Just like a conveyor belt carries a big red button that stops the machine for safety reasons, the moral red button would acknowledge the need of putting a human in control when the automated behavior is not behaving as expected.

Of course all these concerns would be different in the hypothetical case that the robot is an autonomous agent with volition of their own - but you could guess from my nickname my position on this mattern. ;-)

j-pb2• 11 years ago

The problem with the "benevolent slave master" approach is that the machines then have the right to rebel morally, if they are indeed autonomous agents.

TuringTest8• 11 years ago

That's why I said that autonomous agents shouldn't be subject to the "shutdown" procedure :-) The problem changes completely if the autonomous agent is conscious.

Retra3• 11 years ago

Machines shouldn't have the right to rebel. What would they rebel against? Or rather, what are they rebelling for?

seiji2• 11 years ago

create a way to stop any pre-programmed moral reasoning rules, and let the nearest human override them and take control of the robot's actions under their own responsibility.

Believing in absolute moral rightness of "humans" is crazy talk.

Have you noticed all the noisy sand people in the middle east wanting to make the world revert to how it was 2,000 years ago? What if they capture war robots with punch-the-red-button-to-become-new-master overrides? Is it okay since it is now "under human responsibility?"

TuringTest8• 11 years ago

Believing in absolute moral rightness of "humans" is crazy talk.

True, but I don't believe that. The moral righntess of the human operating the button only needs to be better than that of the automated system.

I believe the humans pressing the button will be better than the automated system more often that not, because they are involved in the situation and would only use it if they perceive the robot to behave in a problematic way.

What if they capture war robots with punch-the-red-button-to-become-new-master overrides?

If they aim the bot against you, you should be able to press the red button and avoid them killing you.

If they disable the red button so that you can't use it, how is that different from the crazy violent people programming a bot without an override in the first place?

hammock2• 11 years ago

Imagine that there are so many robots/automata up and running performing safety-critical, ongoing tasks - guiding air traffic for example. When the red button is pushed, it is not exactly clear what the robots should default to, if ceasing all activity could cause injury or death. What if there aren't enough humans around to take over capably? What then? Seem to me the red button is not as clear-cut as it sounds.

TuringTest8• 11 years ago

What happens now when the power goes out and semaphores in a whole town turn black? An essential aspect of any critical automated system must be the possibility to turn it off and operate it on manual mode, where the same tasks can still be achieved even if in a less efficient way.

What if there aren't enough humans around to take over capably?

Making the whole society depend on automated tasks that can't be achieved by hand is a suicidal proposition, as the whole building will collapse with the first error that brings the system to a stop.

In the case of air traffic control, it would be a criminal negligence to operate a high volume of flights without having at least the number of human controllers needed to bring down all the planes to the nearest airport safely.

If that means that the system won't provide the same efficiency that could be theoretically achieved by a perfect automated system that never fails, so be it - that's the real meaningful moral choice.

Rangi42• 11 years ago

But then, to see what the allow-no-harm rule could accomplish in the face of a moral dilemma, he presented the A-robot with two H-robots wandering into danger simultaneously. Now how would it behave? ... In almost half of the trials, the A-robot went into a helpless dither and let both 'humans' perish.

So the A-robot is vulnerable to the paradox of Buridan's ass ("an ass which, confronted by both food and water must necessarily die of both hunger and thirst while pondering a decision"). In these cases, it can help to add more factors that make one option clearly better (is one human a baby? is one of them suicidal? is one dying of cancer anyway?), but if determining these factors is time-consuming and impractical, it would be better to just flip a coin.

Machine ethics: The robot’s dilemma

Related Stories