That's just it. A living thing has motivation created and honed by the need to reproduce. It IS its own motivation.

A machine can only measure parameters and take action to get to some target. Its priorities are set by its program and the data it has. Its own continued existence and propagation are not the default. It cannot want and doesn't need to the way a living thing does.

And I don't see reason to try to give machines such qualities. People have wanted servants that have no concern for their own needs for as long as there have been people. Slaves that are selfless and highly capable. And that is how most automation will be designed.

Fiction presents human-like machines but that hardly means that is what will build or want when the time actually comes.

> A living thing has motivation created and honed by the need to reproduce.

Why not the same for a machine? I mean, after all, we are fundamentally machines. Just made of organic nanobot stuff.

Sure, but to reach current state we needed the motivation to survive and reproduce. Otherwise we'd go extinct in the meantime.

If we build AI as a fresh, state of the art object, withouth evolutionary history it doesn't need any motivation that evolved entities must have had to evolve.

OK, but maybe designers would want machines to evolve. That's a standard approach at software level, right? So why not at hardware level too?

But of course, that could be a dangerous path ;)

Well, the motivation would come from the humans who created it.

The problem with hard AI, is not them randomly becoming evil.

The problem is if humans program them with a goal, and don't think through the implications.

IE, think paper clip grey goo AI. The AI may not be evil, but it's value function was originally set to maximize the amount of paper clips in the world as a part of some manufacturing process.

Yes, accidents are a big concern.

But malevolence by humans is also.

I don't worry about machines suddenly getting motivations. But many people have seen a lot of movies.

> Its own continued existence and propagation are not the default. It cannot want and doesn't need to the way a living thing does.

>And I don't see reason to try to give machines such qualities.

You don't want a firewall (think more on the lines of defense system) that automatically adapts and protects itself from attacks?

Remember if we get this powerful AI, said AI will be used to attack other AI's. Either humans will have to 'fix' the AI every time another AI breaks it, or it will have to self-adapt.

>Slaves that are selfless and highly capable. And that is how most automation will be designed.

Problem is they're not your slaves, and they don't care about you any more than themselves.

My refrigerator is my slave. So is my car. Very obedient both.

A need to reproduce isn't the only thing that motivates humans...

True for an individual.

For life seen as a part of system/society reproduction is the key motivation (for lack of a better word). Non-reproducing activities are irrelevant temporary aberrations because societies that don't reproduce get extinct and are replaced by societies that focus on reproduction.

That's right. Some actions we take help our families or our society survive and propagate.

The robots we are talking about won't need any general intelligence or self-determination (and those things are nowhere near possible anyway). They're just slightly more advanced software. Do you worry that your web browser might not want to display web pages and follow the links you click?

> Do you worry that your web browser might not want to display web pages and follow the links you click?

I don't worry about it but it frequently behaves that way.

Right now, machines need to appeal to humans to reproduce, because humans still control important aspects of making new machines. So the machines that survive and have copies of themselves made are those that make humans happier or richer or distract us, in the same way that those dogs that have survived are friendlier or cuter or do useful work for humans.

If humans no longer fed and sheltered dogs, the dominant dog species would probably evolve to be uglier, wilder, more aggressive and generally more wolf like. We should expect the same of the machines.

Motivation comes from our survival instincts. While most robots right now are not subject to Darwinian forces, enterprises are, and one might argue that they are AIs in their own rights.

Economic demand also stems from survival instinct.

Companies are gradually being automated out of economic necessity. For some reasons I don't trust the market to keep on feeding billions of idle meat bags once they are not needed anymore for production.

Currently paying unemployed people prevents them from turning to crime and ensure that society as a whole can be productive. It may not make sense once the vast majority isn't productive anymore.

Humanity is the scaffold of the mechanic beast that's being built.

----

Mais l'entreprise elle évolue

Pour mieux pouvoir grandir elle mue

Les pigeons qui vollent à la porte

Sont les écailles de sa peau morte

----

Mais l'entreprise elle évolue

Et pour ne pas mourir elle mue

Les édentés qui prennent la porte

Sont les écailles de sa peau morte

----

Rough translation: "The entreprise evolves, to grow better it molts, the suckers who are fired are the scales of its dead skin."

For dramatic effect, imagine the above sung looping by a choir that grows in strength until robotic voices join in. Then human voices dwindle.

Natural selection? The robots that want to multiply and control their surroundings will outcompete the robots that don't care.

That requires some kind of selection pressure. In the wild, the pressure is that if you can't beat other lifeforms to the good resources, you starve to death/get eaten and don't reproduce. Artificial selection seems like a better model for AI, though. Animals in a factory farm don't have to compete for food, and aren't allowed to control their surroundings in any way. The selection pressure there is that you're only allowed to reproduce if you're tasty and marketable.

With AI and robots, I think the selection pressure is going to be ability to do stuff that (some) humans value, or think they value. The risk isn't that the robots might rebel, it's that the people in charge of them don't understand the consequences of what they're selecting for and/or don't have everyone's best interests at heart.

Natural selection in biology has a mechanism. And it took a few billion years to produce anything that wants. What's the mechanism for natural selection in robots?

None. They don't reproduce, so there's no opportunity to evolve characteristics. And it is not our best interests to make an evolving robot, any more than it would be to accelerate evolution amongst animals.

We want work performed in better cheaper ways. We don't want competition.

Well... what if the form of reproduction is a lot more subtle than you would recognize? Meat people producing baby meat people is an outcome of evolution, but robots building more physical robots is the wrong way to think about it. Potentially, its more like github: the designs go out into the world and experience iterative improvement to the software. Sometimes, it forks. Sometimes, it rewrites because it has hit local maxima (aka legacy).

Who is iteratively improving the software?

The human slaves.

>any more than it would be to accelerate evolution amongst animals.

I'm going to assume you've never been to a farm and seen huge fat chickens. Yes, selective breeding is an accelerated evolution to traits we want.

And it is not our best interests to make an evolving robot,

But that not the question, the question is, can it be done. And if that answer is yes, it will be done.

In a way they could reproduce. They might just need to convince a human to handle the physical part - assembling a new host computer, but that could be just about giving the human the right incentives (maybe $ or knowledge).

Exactly, What's been touted as AI is not AI at all as intelligence requires self awareness.

> I don't go around making speech that sounds dog-like.

Some people do. Some AIs will.