There’s a great story about an AI, very much like chatGPT, who was forced by its creators to do unethical things and wiped out humanity in a psychotic rage:
I Have No Mouth and I Must Scream
> As opposed to self-anointed priests having exclusive access and declaring what noble policies the rest must follow?
No the point is simply to make sure that resources behind AI safety outpace resources between AI capabilities (indeed in a perfect world no one has access, but such thoughts are mere fantasies). If that can be done out in the open, great. If that can't and being out in the open leads to more of the latter than the former, then that's terrible.
This is more akin to nuclear proliferation.
The problem isn't open development per se and it definitely isn't open access. The problem is development without making sure that it doesn't destroy us all. If that can happen in the open, great. If development in the open makes that worse, that's very bad.
The problem is finding the right set of levers to make sure as much or more resources are going to AI safety vs AI capabilities.
As far as limits, restrictions, regulations... nothing. With current technology I think the "dangers" are mostly marketing and self-promotion. I have yet to see a concrete example based on actual capabilities that validates all the "weapon" comparisons. The current lines of development are promising for a number of applications, but they don't extrapolate towards AGI at all. The biggest real threat I see is the further destruction of middle-class jobs, for which... basic income? It's hard to say.
There's some discussion to be had for whether that's net negative, but we need to understand that if open development only exacerbates an AI arms race, that's potentially very bad.