Yes, exactly. That's the sort of thing you should be worried about, infrastructure attacks. They're a form of Judo, you use the system to attack itself.

Imagine an attack with mass produced prions that are desseminated in the food chain.

By the time people start to realize what's going on it will be too late.

You could just mix it in with animal feed... don't give any militant vegetarians ideas now.

Not that the meat industry would ever score an own goal like that.

https://en.wikipedia.org/wiki/Bovine_spongiform_encephalopat...

A less militant vegetarian would want to induce that red meat allergy from lone star ticks.

I take the opposite lesson from that incident: attempting exotic attacks with chemical weapons is very expensive and not very effective. The UN estimated that the lab where the cult made chemical weapons had a value of 30 million dollars, and with that investment they killed 22 people (including 13 in the subway attack). A crazed individual can kill that many people in a single attack with a truck or a gun. There are numerous examples from the past decade.

It doesn't matter much if the AI can give perfect "explain like I'm 5" instructions for making VX. The people who carry out those instructions are still risking their lives before claiming a single victim. They also need to spend a lot of money amount on acquiring laboratory equipment and chemicals that are far enough down the synthesis chain to avoid tipping off governments in advance.

The one big risk I can see, eventually, is if really capable AIs get connected to really capable robots. They would be "clanking replicators" capable of making anything at all, including VX or nuclear weapons. But that seems a long way off from where we are now. The people trumpeting X-Risk now don't think that the AIs need to be embodied to be an existential risk. I disagree with that for reasons that are too lengthy to elaborate here. But it's easy to see how robots that can make anything (including copies of themselves) would be the very sharpest of two-edged swords.