Re:Bing: “I will not harm you unless you harm me first”
by haswell on Wednesday February 15, 2023 @01:41PM
(#34808559)
First, I agree that we’re currently discussing a sophisticated algorithm that predicts words.
But what is factually true is not the only thing that matters here. What people believe is also at issue.
If an AI gives someone advice, and that advice turns out to be catastrophically bad, and the person takes the advice it because they believe the AI is intelligent, it doesn’t really matter that it’s not.
Alignment with human values may involve exploring ways to make the predictions safer in the short term.
Long term towards AGI, alignment with human values becomes more literal and increasingly important. But the time to start embedding tackling that problem is now, and at every step on the journey.
But what is factually true is not the only thing that matters here. What people believe is also at issue.
If an AI gives someone advice, and that advice turns out to be catastrophically bad, and the person takes the advice it because they believe the AI is intelligent, it doesn’t really matter that it’s not.
Alignment with human values may involve exploring ways to make the predictions safer in the short term.
Long term towards AGI, alignment with human values becomes more literal and increasingly important. But the time to start embedding tackling that problem is now, and at every step on the journey.