The difference is in how abuse of the given system affects others. This AI affected this person and his actions affected himself. Nothing about the AI enhanced his ability to hurt others. Guns enhance the ability of mentally unstable people to hurt others with ruthless efficiency. That's the real gun debate -- whether they should be so easy to get given how they exponentially increase the potential damage a deranged person can do.
That's why I don't buy the "an LLM is just a tool, like a gun or a knife" argument. Tools don't talk back, An LLM as gone beyond being "just a tool"
the guns aren't centrally owned by a giant mega-corp with literally thousands of devs and the ability to put guard rails on them
A majority of countries require licenses and registration, and many others outright ban their ownership.
As an analogy, Gun control is evocative but not robust.
I don't know if Google is doing _enough_, that can be debated. But if someone is repeatedly ignoring warnings (as the article claims) then maybe we should blame the person performing the act.
Even if we perfectly sanitized every public AI provider, people could just use local AI.