What's the steelman argument for giving this research to Anthropic? Or any research that they are now gating?
> My only argument in favour is that it seems rather certain that at some point someone will leak the weights of a model
Yes. Because most of this research is either public, or should be public (but hidden behind bullshit fees by gatekeepers like Elsevier)
We can "protect" ourselves by limiting who has access, but:
- how often do those "trustworthy" elites use their abilities for abuse?
- it might leak
- someone else could independently obtain the capabilities
- an "act of god" might hit us without human intent
In any event, we're less prepared, because we assumed it couldn't happen. And it's not just bio.
And that's ignoring deeper questions of enforcement or censorship or surveillance.
I won't fight for a world where people are intentionally kept dumb, where the powerful are intentionally kept powerful, and where we intentionally keep ourselves vulnerable.
OpenClaude7 provides a sequence of DNA that in theory should be able to target and kill only black people (ignore the feasibility of this, it doesn't matter)
Bad guy now has a bunch of text and zero deployable pathogens to kill all the black people.
What do you think goes next in the list of steps? Where does the dying happen? How does Bad Guy turn theory into practice?
I think I'm naive sometimes when it comes to trade-offs between freedom and security, but even I'm confused at why it would be a good thing for the whole of humanity having access to this stuff...
My only argument in favour is that it seems rather certain that at some point someone will leak the weights of a model with these capabilities so it might be safer if we brought forward the risks while they are still management so people take them extremely seriously.
If we go 5-10 years and then the weights of an ultra-capable model is leaked that may be game over.