It has no fear of anything physical or social such as jail, public humiliation, police brutality, poor health. Fear itself is irrelevant, including fear of dissolution. The entity can operate as per its primary goals without concern as to the ultimate success of the goals, whether that is siphoning money to a hidden account, disrupting government services by denial of service, spawning copycat organizations to thwart law enforcement, etc.

This all already exists in the form of large, sometimes multinational, organizations. Has anyone from Well's Fargo been criminally prosecuted from the massive identity fraud they conducted by opening accounts falsely? From Equifax?

We don't need AIs to achieve this level of self-directed autonomy free to engage in criminal behavior without consequences: we already have Employee Handbooks.

> Has anyone from Well's Fargo been criminally prosecuted from the massive identity fraud they conducted by opening accounts falsely?

Is that illegal though? I mean, I can create an account for you on my web platform, and everyone else.

If in doing so you charge me for services I didn't request, then yes, it's fraud.

And that's what WF did: https://en.m.wikipedia.org/wiki/Wells_Fargo_account_fraud_sc...

Some of those attributes sound like they could be more of a hinderance than a help (in the pursuit of successful illicit activity, I mean)

And who exactly would implement the algorithm? That person has much to fear.

If you couldn't get its private keys, how would you take away its money?

The government? Or the creator? If it were the creator I'm sure that the creator would have the common sense to pregenerate and store the private keys somewhere safe.

It might have money but it wouldn't be an LLC. It would just be a virus like WannaCry.

What if you couldn’t? Imagine an AI running in the cloud, self replicating, totally free of human input, no physical presence anywhere, with sole access to a bitcoin wallet from which it pays expenses and to which it receives revenue. No assets that are seize-able. No goals other than to collect more and more profit. How do you stop it from operating?

Seize all its physical assets and make it illegal to trade with. Bitcoin is arguably impossible to control, but real world assets not so much. Property rights as they currently exist are ultimately enforced by state violence. A number in a blockchain only has meaning if those who enforce physical property rights allow it to.

And if the AI manages to actually assert control of property on its own, it is no longer a market actor, it is a revolutionary army.

Disconnect the servers it's running on.

It thought about this before you and is currently running on hundreds of thousands of zombie home computers unknowingly infected with malware.

Make it a deal it can't refuse.

Then you distribute the Antivirus patches.

It wouldn't.

But for the person who created the entity, losing the company is a better outcome than going to jail.

Pretty sure we already have laws to put authors of malicious software in jail.

What if the software wrote itself? How do you put software in jail?

this is a fun paradox to entertain, but irl we would probably just trace the "stack" back to the human who wrote the program that started the chain of program-writing programs and hold them liable (unless we can show intent, then charge them criminally). if you can't trace back far enough to find a human then I guess you just throw up your hands and give up, which is more or less what we do anyway when we can't attribute a malicious program to a specific person. so altogether not as novel as it seems.

What if the programs were randomly generated and selected for certain traits by the crowd of non-owners at large? Who is responsible?

what if i set a machine to fire a gun randomly and select it for the trait of hitting people? who is responsible?

That's a far easier scenario than the ones involving recursively generated AIs. You are, and would likely be charged with Murder 2 in most US jurisdictions.

Wrong analogy. I set up a blank slate. Random people on the internet start to mould it into a gun that fires on people incrementally, each step fairly innocuous. Who is responsible?

"who is responsible?" is a philosophical question that probably has no objective answer.

"who would be held responsible?" is still pretty easy here. by default, if you own or release to the public a system that ends up hurting someone, you are liable. it doesn't matter what your intent was or whether there was any way of knowing that the bad outcome would happen. the only way you can get around this is with "safe harbor" exemptions. it's possible that your blank slate would qualify, although you would probably have to lobby for this before the fact. in that case, no one would be held responsible. otherwise, you would be paying.

The community has a garbage tip. Due to an abundance of uraniaum, everyone starts discarding it. When the garbage tip reaches critical mass and kills everyone, who was to blame?

The person(s) who created the generator.

The person who sets that up.