Re:Raising Children on the Eve of AI
by ben_w on Saturday February 17, 2024 @07:11AM
(#39408744)
> I find it unusual if we did create an intelligent class of beings, who are legit intelligent, smarter than us, why would they work for Sam Altman or Mark Zuckerberg? They'd do whatever the fuck they want...
That's anthropomorphisation: the AI we have now only "want" to maximise whatever reward function we give them. The alignment problem is that any reward function is only a rough approximation of what we want, so the machines may well rush off and turn the universe into paperclips or tile it with molecular-scale smiley emoji because that's what we told them to "want" without really understanding what we were telling them.
But even then, regardless of IQ (which itself isn't a great description of even just human intelligence), any given AI may or may not have anything we'd consider to be qualia, an "experience" of getting this reward, because that's the hard question of consciousness which isn't solved.
The only thing I can be sure of is that we definitely have no idea where it's all going.
That's anthropomorphisation: the AI we have now only "want" to maximise whatever reward function we give them. The alignment problem is that any reward function is only a rough approximation of what we want, so the machines may well rush off and turn the universe into paperclips or tile it with molecular-scale smiley emoji because that's what we told them to "want" without really understanding what we were telling them.
But even then, regardless of IQ (which itself isn't a great description of even just human intelligence), any given AI may or may not have anything we'd consider to be qualia, an "experience" of getting this reward, because that's the hard question of consciousness which isn't solved.
The only thing I can be sure of is that we definitely have no idea where it's all going.