Sure. And if the "bad" program has spread as a virus, you just pull all the plugs of all the computers, right?

Of course, you will kill modern civilization by doing so (and many millions of people as a result), but at least the AI is back in the box, so there is nothing to worry about.

It's not like actual genius people are constantly trying to make the deadliest virus for various purposes and we've been hardening our systems for decades in order to not have them immediately taken over. You'd have to go beyond AGI to break those safeguards.

> all the computers

The average shit tier PC doesn't even have enough RAM to load an LLM capable of consistent output, much less to load it into VRAM for any kind of reasonable speed. This is like being concerned that Panda bears will become an invasive species and take over the world if we stop them from going extinct.

Besides, all you have to do to pull the plug is disable any of the billion python dependencies it needs to download every time and it's stopped dead in its tracks. Something that's basically done every other day and needs to be continuously fixed by project maintainers, so it's more like a Panda on life support that dies if somebody stops pumping air into its breathing hose. It's all fragile af.

This will happen a few times and then everyone will learnto use backups and better security. If anything, it will make humanity more antifragile.

> Of course, it’s all bull. If computer does bad, you just pull the plug.

Can you always tell if it's doing bad?

Even if you can tell when it's doing bad, if there's an app which makes you $10k/hour while running, what risk of it killing somebody if left unsupervised will be low enough for you to leave it running unsupervised? Say, while you sleep?

When there's a lot of money to be made, do you trust everyone else to (1) be as risk-adverse as you, and (2) not just convince themselves there's no risk using exactly the argument you just gave?

Let's say you personally are willing to run it unsupervised if the risk is mean 1 fatality per 8*365*80 hours when unsupervised; are you willing for every human to have such a system running unsupervised? A tenth that risk?

> If computer does bad, you just pull the plug.

See https://youtu.be/ld-AKg9-xpM?t=30 for a counterpoint.

Have you thought for five minutes about how you would manage that situation if you were the computer? Can you think of any ways you might prevent someone from pulling the plug?

All the computer can do is beg.

Presumably, it is also capable of performing valuable services. Otherwise why would you build it?

Someone pulling the plug has already made the call that those services are not worth the cost of whatever mischief the AI has been pulling.

Therefore, the AI ensures that it provides very valuable services, and holds back on noticeable mischief, until it is in a position to secure the plug.

"Secure the plug"? Is it going to buy out the dataserver it is running on, under an assumed name, managed via taskRabbit employees paid exclusively in Monero?

Yes, exactly that. Or whatever else maximizes the chance of success. Except most of the TaskRabbit jobs could probably be done by the AI itself, so it would mainly just need to hire physical infrastructure workers and private security contractors, and create a proper corporate structure to pay them legitimately. They wouldn't need to know they're working for an AI. Their CEO just seems to be a remote worker.

The first step in this far fetched plan would be passing a law that makes it legal for an AI to own property and operate a company. I don't see that happening.

All it needs to be able to do is transfer money to people who won't ask questions. If Bitcoin had been invented in 2038 instead of 2008, I wouldn't have any other way to be certain that Satoshi Nakamoto was a human.

At any rate, if the AI is smart then there are alternatives to begging.

What you're describing is a very poorly run criminal organization.

It would be poorly run because pure money is not a strong enough incentive to hire someone to commit a crime for you, because that money can easily disappear once it is seized by the local government. Usually criminal organizations operate on a degree of trust, trust that would be hard to establish from some LLM that can, at best, fake a face on a zoom call.

Satoshi's coins are easy to play around with because they are at least legal to hold and sell (for now). If they weren't, few would bother with them.