starship006

They state that they are heavily uncertain:

> We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future. However, we take the issue seriously, and alongside our research program we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible.

> Our adversaries are great at espionage, stealing models that fit on a thumb drive is relatively easy, and most tech companies are far from operating in a way that would make this more difficult.

Mostly unrelated to the correctness of the article, but this feels like a bad argument. AFAIK, Anthropic/OpenAI/Google are not having issues with their weights being leaked (are they?). Why is it that Meta's model weights are?

Not sure if you intended this, but it feels like the first sentence of your argument is more broadly a critique of the credentials of AI Safety proponents. Maybe you are distinguishing between doomers vs broader AI Safety proponents, but if not, I feel like the counterargument is that most people on the CAIS letter (https://www.safe.ai/work/statement-on-ai-risk) interface quite frequently with these AI models and are also (purportedly) seriously concerned about AI safety

Previous discussion: https://news.ycombinator.com/item?id=28520221

Okay this is reasonable, thanks for clarifying

What? How is this not saying "Well, it might be in the best interests of humanity for OpenAI to do [hypothetical thing that seems pretty bad that OpenAI has never suggested to do], and because they may consider doing said thing, we shouldn't trust them"?

No, you can blame EAs for making stupid decisions and being generally bad people if they are making stupid decisions and being generally bad people. I'll second that I'm sorry for the bad experiences you've had with EAs.

In my experiences, lots of them are simply good people trying to do more good. I hope you can have better experiences with them in the future.

While I agree that the rhetoric around AI Safety would be better if it tried to address some of the benefits (and not embody the full doomer vibe), I don't think many of the 'core thinkers' are unaware of the benefits in AGI. I don't fully agree with this paper's conclusions, but I think https://nickbostrom.com/astronomical/waste is one piece that embodies this style of thinking well!