Risks to physical danger evolve all the time. It’s not a big leap from AI generated this script to a fatal bug is nefariously hidden in the AI generated library in-use by mission critical services (e.g. cars, medical devices, missiles, fertilizers).

AI is now poised to make bureaucratic decisions. Bureaucracy puts people in physical danger every day. I've had medical treatments a doctor said I need denied by my insurance, for example.

For somebody from another country this sounds insane..

Others siblings have good replies, but also, we regulate without physical danger all the damn time.

See airlines, traffic control, medical equipment, government services, but also we regulate ads, TV, financial services, crypto. I mean we regulate so many “tech” things for the benefit of society this is a losing argument to take. There’s plenty of room to argue the elsewhere but the idea that we don’t regulate tech if it’s not immediately a physical danger is crazy. Even global warming is a huge one, down to housing codes and cars etc. It’s a potential physical danger hundreds of years out, and we’re freaking out about it. Yet AI had the chance to really do much more damage within a much shorter time frame.

We also just regulate soft social stability things all over, be it nudity, noise, etc.

Let me recalibrate. I'm not arguing that there technology or AI or things that don't cause death should not be regulated, but I can see that might be the inference.

I just think that comparing AI to nuclear weapons seems like hyperbole.

Why is it hyperbole? Nuclear weapons and AI both have the capacity to end the world.

Private citizens and companies do not have access to nuclear weapon technology and even the countries who do are being watched like hawks.

If equally or similarly dangerous, are you then saying AI technology should be taken out of the hands of companies and private citizens?

For the sake of argument, let's say yes, AI should be taken out of the hands of the private sector entirely.