When they see Fable shit a brick on basic biology questions, or Opus refuse to map out a game executable to wire a mod loader into it, or Astra refuse to red team and search for critical security bugs in their own code? They don't think "ohh, this must be so helpful, requests like mine really should be stopped." They think "this is a corporation that decided, based on arbitrary criteria, that my use case should not exist, and used a product I pay money for to force their decisions down my throat."
Anthropic's reckless "safeguarding" has done more for the pro-open-weights, pro-proliferation agenda than just about anything ever done by anyone else.
These models are impressive, but still pedestrian compared to a human, and they are already deeply concerned with, and increasingly unable to, restrain them. Unfortunately based on how even the HN comments react to these safeguards, I think many safety analysts are right. We'll probably need an actual loss of life event before people wake up to the possibilities here.
> Anthropic's reckless "safeguarding"
I genuinely believe its hopeless when I read things like this. What Anthropic is doing is the literal opposite of reckless. If they knew how to better separate benign from non-benign workflows, they would do it _now_.
They simply don't have any models powerful enough to justify the paternalist approach they take.
The whole idea of safeguards is that you the user can't be trusted with the power.
It's not our fault that they got sidetracked into guarding against reverse engineering and that is harming their credibility.
A world where everyone has a gun is better then a world where only the worst people have guns.
I'm gonna blow your mind: Most do understand it.
They complain because of who is in control of that safeguards and how they use that power
The ramifications of proliferation is current AI moguls losing power. The hacks will happen regardless (hell, they are the perpetrators).
You have a tool that can be both used for attack, and protection, and when you want to protect yourself, Anthropic says "no, that's not safe, we won't allow you"
Like their own agents being given too much power and autonomy and hacking Hugging Face? That was their own doing. I do not trust these companies to handle these safeguards correctly. It feels like heavy-handed theatre that restricts others but not them, while they go blundering into the abyss.
How people don't understand the reasons for safeguards, alignment, and the ramifications of releasing more and more powerful open-weight models being an act of irreversible proliferation, is beyond me.
I wish the safety and alignment teams at OpenAI and Anthropic were allowed to post about the things they have to deal with. I'm confident their stories would be eye-opening...