I don't think this is comparable. Physical weapons can't be used as protection; you can only use them for counter attacks.

AI on the other hand side can be used for protection—because it's not a weapon per se.

A gun can itself be used to block a bullet, or to shoot down an incoming bullet, but they're both near-impossible to pull off. It's much easier to use the gun to shoot the attacker before they shoot you, or even better by its mere presence deter the attack in the first place.

This makes economic sense: it's expensive to armor every vulnerability and cheap to exploit just one.

I don't see a reason that AI would be different from guns or missiles in this regard: offense (or the MAD threat of it) is the best defense.

Your (oblique) comparison to gun control is interesting... but practically, controlling a technology which is just a special kind of number-crunching is vastly harder than controlling a technology which must be forged from steel in a special factory.

This was discussed in another place, the "then shouldn't everyone have nukes?" argument https://news.ycombinator.com/item?id=35159757

I don't stand anywhere too concretely on the issue yet, all I know is that AI is dangerous in a different way than guns or nukes, and therefore should probably be treated different. There's just something different about software, maybe because no matter what at the end of the day, if it wants to actually kill someone, it has to interface with the real world, and so my idea of "just stop anybody that ever tries to build nukes" might be good enough? That, plus, hardening critical infrastructure against network attack (or just airgapping it... or making it analog!)

I'm wondering how much this has been written about already, if anybody has some more in depth musings on the subject I'd be interested. I have yet to read Kurzweil, maybe he's covered it.

I'm interested in your observation that this is mostly coming from the USA: though I was born and raised in America, typically I align strongly against American values, though I can't deny that they form the basis of my subconscious. My belief though is that this ideology comes from a combination of reading people like Doctorow, and reading Karl Marx, who argued in favor of an armed proletariat that should resist any efforts by anyone to disarm them. In conversations with anarchists in Greece as well as all sorts of folks at places like Defcon, I've come to understand that the distribution of the means for resistance is a generally good thing, and this is where my belief stems that this technology should be freely available.

I don't know how well that applies to the USA though which obviously has a horrifying gun violence epidemic. Something weird is going on there though, other countries with a proliferation of guns (though not to that level...) don't have the same issues.

Thanks for the other discussion!

> if it wants to actually kill someone, it has to interface with the real world

And researchers already tested for GPT-4 to be put online. That is already being interfaced with the real world, through the minds of anybody the AI may talk to/be made to talk to (see how social networks can be used to manipulate towards a very real outcome, if only from doxxing to "cancelling" to peoples' Spring revolts). Shaping people's minds is a powerful thing. And that's the most likely use case to me at this point, but who knows what it will be in the end.

Once online, we could easily envision that an AI may act on some purpose[1] shop, put orders, ships stuff with instructions and incentives to real people, or even be plugged into some real-world APIs.

[1] not purpose in the sense "is it conscious or not". It does not matter if it's conscious or not, if it's AGI or not. What matters is if/that it may function in a systematic and working way following an impulse/stimuli.

It may not even be about killing or destruction really, but it could be about shaping some society policies or self-understanding, which would have consequences down the line too.

Distributing power in society is a good thing, as long as its usage stays at an individual human scale.

When the first mover advantage has the potential to deeply harm people or society (fire weapons, nukes, cars, AI that redefines what truth looks/sounds like or that can act in a non-deterministic way based on a per-construction biased-dataset, in a fraction of a second), there's a responsibility that has to come with it, and there's a custom to set rules to control who has access to it, for what purpose, to train them and to have a social/political control over them.