> At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up.
That's the point, it is not possible. The premise is based on a paradox and you can't throwaway containment and have alignment that remains defined by original intent.
Re:7 Weeks AI Alignment Curriculum
by abecedarius on Sunday March 05, 2023 @10:14PM
(#35037130)
If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that. (Note that a smart human on much faster hardware is smarter-than-human. This is not a claim that AGIs must be mere faster humans; it's one example to point out a difficulty for any such argument.)
But I think you gave a false impression of others at the start of this thread.
> If you want to argue that it's logically impossible for a smarter-than-human intelligence to be created with values aligned with human ones, then cool, do that.
That is precisely what I have stated.
> But I think you gave a false impression of others at the start of this thread.
How so?
- Creating an AGI whose goals are compatible with ours.
- Creating an AGI with inscrutable goals and safely confining it.
They're not the same!
In whoever you've been talking to I can't recognize the rationalist scene I've known since it grew out of the 90s extropians. "AI boxing" was a live argument around the early 2000s. Not now.
Online you often see people in that community saying that slower progress in AI capabilities would be a very good thing. At the same time, yes, they do want us to achieve AGI/ASI as soon as possible, provided we don't screw it up.