Besides the other comment which is quite correct, we're going to suffer from B far before we will suffer from A.

That is a valid reason to worry about B, but not a valid reason not to worry about A!

Because the AI controlled by humans is a tool, a technology, that can be used in various ways.

Are you worried about knives in general or are you worried about a knife in the hands of a particular person in a particular moment?

Okay, but that assumes that control of AI is as trivially easy as controlling a knife. Given that this is one of the key unknowns, assuming that the AI never does anything that its controller doesn't intend is begging the question.

That aside, I can also think of many cases where I would feel less safe around a machine that had nobody in control of it, than one being adeptly controlled by an operator.

One important thing is that nobody has shown an actual existing AI to be not controllable, nor a way to create one. Until then it's an imaginary, non-existing risk. Of course as a fear and anxiety-based society we can also worry about those, but I think we should worry more about existing risks.

AI doesn't act by itself and worst case you could unplug the bloody thing (SF notwithstanding). But the guys with the guns are the same guys with the guns that have been in charge for as long as there are guns (with some rare exceptions) and I really don't think the existence of AI will change human nature.

So AI will be used as a weapon long before it will be a standalone threat. And I like to focus on the things that are more immediate because that's where the threat is most likely to materialize and have effect on me.

> AI doesn't act by itself

It can be made to quite easily.

> and worst case you could unplug the bloody thing

Not necessarily.

> But the guys with the guns are the same guys with the guns

Well, unless they are autonomous AI drones with guns (or missiles), in which case they are different “guys” with guns.

Sure, I've seen those movies. But today it's the guys with the guns/drones/grenades/missiles.

I'm worried about stable dictatorships far more than I am about AGI taking over the world and offing humanity because stable dictatorships are already using every weapon they can lay their hands on to control the population and AGI might just tip the scale to the point where they become eternal. And regular AI (the kind that we have today, because AGI is simply not yet a thing that has been achieved afaik) is more than dangerous enough in that context.

> But today

There's no question that this is a tomorrow-focused concern, just like climate change or Kessler syndrome, et cetera.

And today's concerns are important too. Nobody is dismissing them. But tomorrow comes quite quickly these days.

You're allowed to be concerned about both.

Depending on where you live climate change may very well be a today thing.

Kessler syndrome not so much.

Guys with guns and missiles: very much a today thing.

I mean, via Starlink, Musk is working pretty hard on making Kessler Syndrome a today thing as fast as possible.

Not in LEO.

>> AI doesn't act by itself

> It can be made to quite easily.

Not really, no. You can of course create an acting-by-itself bot but it will run into limits very very soon, because the world model and planning parts of AI are in their infant stages, if that.

The concern is not about GPT-4. It's about what 10 years of research gets you to when, five years ago, it seemed completely apparent that even GPT-4 was 50 years off.

Ai is having explosive growth and the hype around it and research is definitely at a high. You could say we are in a "bullish market" when it comes to AI R&D ;). Assuming that this persists for 10 more years or even beyond that seems like a bad bet but I guess we'll see.

GPT-1 was 5 years ago, GPT-2 was 4 years ago and by that time it was quite clear to some that this will scale soon.

But I'm not talking about the GPT-4 either. Many decades and billions poured into research still didn't give us autonomous driving and there is no clear path how that should happen. So I'm not really holding my breath for anything even more capable.

> worst case you could unplug the bloody thing

Okay, now you're failing at putting yourself in the AI's shoes and asking how to win from its position, even using your own intelligence.

Have you even considered how you might win as an AI, knowing that you can be unplugged? It's definitely doable.