It's a critical problem like when a drug dealers supply kills someone and they get a bump in business because they're selling "the real deal"

Irrelevant to your point, but drug users dying is more often the result of a dealer cutting their supply with something dangerous than it is the result of purity.

Right, and an LLM being able to "escape confinement" is more likely to be poor confinement or a PR stunt than a "too powerful" llm. Same situation

No, not really as almost no one runs LLMs in confinement once they are released and the capability is a jailbreak away. Poor confinement is irrelevant, if I use your model to check the security of my code and it hacks into the company that makes libraries used inside of it it's a huge problem.

Second this. Lots of fentanyl overdose are caused by accident, not because the buyers are buying them. It is extremely dangerous

What matters is the spin they give in the media. And so far the winning story is “our model is so powerful it can do this”. How many people dig into it and what independent data do they even have? They comment on the title. And so the image of this superhuman AI from OpenAI propagates.

We have no reason whatsoever to trust anything OpenAI says. Except to assume it will be self serving. As the article points out, ChatGPT 2 was also “too dangerous” and we can all agree even for the time this was just marketing. They rinse and repeat the same technique whenever they need to draw attention and money.

In any other field you’s expect independent testing, peer reviewed studies, but here it’s just “company who makes product says product is fantastic, surpassed all expectations”. They wouldn’t lie to us, would they?

The word "marketing" in this specific context is missing the point. OpenAI (and Anthropic) needs to kneecap open source competition with burdensome regulations. Without that, their valuation simply makes no sense.

This stunt was different from their graden-variety scaremongering. Its timing was precisely calibrated with upcoming release of Kimi K3 and the Nvidia-led open model coalition. To me it seems transparently related to the protections OpenAI are lobbying for (and they are citing this incident as support).

I can only hope that the clumsy way it came off, and the fact that only an open weights model could defend HuggingFace, will blunt its effectiveness.