Compare with nuclear weapons, which are way worse and have a lot of support for a ban, but there's no serious hope of that ever happening, because they are really good at killing people and destroying stuff.
Seems to me that a ban is only realistic if the stuff doesn't work well.
Against a modern military equipped and prepared to withstand them, perhaps. Against a civilian population or a less well-funded force, not so much. This is doubly the case when the attacker is a non-state agent (like a terrorist). Remember for instance the Sarin gas attack in the Tokyo metro by the Shinning Path [1]. An air force bombing a major city with chemical weapons would also cause vast casualties.
I think what you're trying to say is that chemical weapons are primarily designed to be terrifying with lethality considerations coming next. However, you could argue the same about nuclear weapons. After all, the attacks in Hiroshima and Nagasaki were not acutally meant to wipe the Japanese apart, only to scare them into submission. And nuclear deterrence works on the principle that an enemy won't dare attack a nuclear power.
In general, you could think of all weapons ever as aimed primarily at the morale of the enemy rather than their life and that thought may have some merit. After all, an army that breaks and runs is beaten faster and with fewer losses to the other side than one that fights to the last [2].
However, the important point to keep in mind is that weapons, even when they're primarily designed to be terrifying, are designed to be terrifying in the manner they kill. So I think you'll find that most people would agree that chemical weapons should be banned because they kill in a horrible manner, not relative to the number of casualties they may or may not inflict.
I do think you underestimate their potency btw.
__________________________
[1] https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack
[2] Cough cough. In war games, certainly.
I guess you could think if we can't just get rid of war (which is always terrible), there's no reason to have international laws and agreements limiting certain kinds of killing and other harm as off limits and unacceptable.
Apparently our current leaders agree with you.
The position of the Canadian government is that the development of autonomous weapons is already illegal under Article 36 of the 1977 Extended Protocol 1 to the Geneva Convention of 1949.
Could also forbid self-replication of weaponized robots.
Not sarcastic.
Pacifism is an unstable configuration; non-aggression is.
Laws as written have all kinds of obvious loopholes, but they are intentionally vague enough that a judge can look at your "obvious loophole" and decide that you are in violation anyway.
That Google, Apple, Uber et al pay so very little tax suggests that that is an optimistic appraisal.
Killing AI is not the same problem with nuclear weapon. Because AI weapon is far easier to make compare to nuclear one. Technically, you can do it right now with your DJI drone, all it takes is just to program it to do some automatic control.
So I'm not quite optimistic about any attempt on limit those kind of weapons base on current condition we are in.
Yet, people stopped mentioning this kind of warning even it was already there for way over half of a century.
Reason of we successfully limited use of nuclear weapon is because, well, the US tested it on Japanese, saw the devastating result and thinking "What if the Japanese (Or Russian, Chinese etc) one day use it on us?".
I would guess only similar event can trigger everyone on earth stand in line and effectively ban those AI weapons all together. That's AFTER we all actually be seriously hurt by it.
It's just like the warning of globe warming, the math is there, proved by observation, somebody out there still believe we will be fine because the whole globe warming thing is just a conspiracy made by China.
We're only safe, because crazy or truly evil people are rare.