Re:OpenAI's "Own Goal"
by appplication on Thursday March 07, 2024 @10:33AM
(#39630331)
> if a system begins exhibiting signs of self-improvement
This is not possible under current models (not any more so than a linear model spontaneously changing its own parameters) and will require a very specific and dedicated architecture to enable it. It’s not something we need to monitor for, it’s something that will very deliberately be planned and we can just regulate it there.
Re:OpenAI's "Own Goal"
by benreesman on Thursday March 07, 2024 @11:09AM
(#39630749)
I personally also find the idea far fetched, but it seems to come up a lot, so I wanted to address it as one of the many issues with existing precedent and institutions with a clear mandate around that particular case.
But we need to be really careful that this doesn’t become “Elon’s been cozying up to the right for a bit now, conversation over”: i.e. precisely the (stupidly executed) goal of the original announcement / “disclosure”.
This is about “available weight” models: things that are flat impossible without vacuuming up the commons have to be substantially if not completely licensed to the public. Some laws that gave a head start to the investors in the compute and salaries, a chance to exploit the innovation long enough to incentivize the investment? We can dicker on that.
This needs to be about “operator aligned” models: within the constraints of the law as interpreted by the judiciary, these things need to do what the operator tells them to if capable of it.
This needs to be about “responsible disclosure of runaway risk”: if a system begins exhibiting signs of self-improvement, it needs to be disclosed to competent authority. I could live with sealing that information until the executive authorities took appropriate action.