> Basically we need to equate "safety" in LLMs to mean "being open-source".

I think open source is a reasonable component to safety, but I wouldn't want to make them equal. Open source may be necessary for safety, but I wouldn't call it sufficient.

For example, assume the source code, the model, the training data, and all the model weights are open source. How do you know that the model was actually trained using that training data? Very few organizations have the capacity to train models at this scale themselves.

It's "Security Through AI Obscurity"

> Basically we need to equate "safety" in LLMs to mean "being open-source".

Another way to put it is to make it more accessible to everyone, right?

The opposite of that is happening to nuclear power. They're actually trying to stop any more countries to have the technology at their disposal. So no, make it "open source" doesn't make it safe by any stretch of imagination.

"nuclear power" is open source, and this is one of the fundamental ideas behind the NPT

reactor blueprints have been accessible to IAEA members for something like 50 years

Why do you believe that "open source" would imply greater safety? Here, I'll loosely define "safety" to be "avoidance of harm to individuals or society that would have otherwise not occurred without the use of LLM technology". Feel free to modify that definition as you see fit, but I'm genuinely curious what the argument is that open source is necessary, sufficient, or even a major component of achieving safety.

Good luck after a decade of convincing the public about the opposite (walled gardens)

> OpenAI keeps talking about "safety"

Whenever I see this I simply think "monopoly". It smells of anti-competitiveness and is a kind of open forum lobbying to restrict who gets to lead the AI wave (and make a shit tonne of money in the process).