Would you have called the possibility of large language models helping millions of people "unhinged" a few years ago as well?

No? That’s been the goal of NLP and information retrieval research for decades.

The goal is also to develop systems that are significantly more capable than current systems. And those systems could be misused when terrorists gain access to them. What about that is "unhinged"?

It's unhinged because one could make slippery slope arguments about any technology killing millions of people.

In the cold war era, the government didn't even want cryptography to become generally available. I mean, what if Soviet spies use it to communicate with each other and the government can't decode what they're saying?

People who are worried about technology killing people ought to focus their efforts on the technologies that we actually know kill people, like guns and cigarettes. (Oh but, those industries are donating money to the politicians, so they conveniently don't care much.)

Cryptography can't be used to produce weapons of mass destruction. It's a purely defensive technology. Engineered superviruses are a whole different caliber.

Wrong, cryptography is used to conceal the communications of people building nuclear weapons. It still should not be regulated.

Cryptography is just as useless for creating nuclear weapons as it is for creating bioweapons. AIs that help make WMDs are on a very different level.

I'm fairly sure I'd describe all terrorists as unhinged.

Also, we've got plenty of examples of people not reading the instructions with AI (those lawyers who tried to use ChatGPT for citations), and before that plenty of examples of people not reading the instructions with anything and everything else. In the case of terrorists, the (attempted) shoe bomber comes to mind, though given quite how bad that attempt was I question the sanity of everyone else's response as many of us are still taking off shoes to go through airport security.