News for hackers. Stuff that matters.

A comment on Past Performance is Not Indicative of Future Results (2020). Parent comment

Re:Past Performance is Not Indicative of Future Results (2020) by exporectomy on Saturday July 31, 2021 @09:20PM (#28022924)
People tend to predict simple technological substitutes for human tasks rather than novel things. I suspect we won't get artificial humans because we'll end up not actually wanting that and getting something better instead. Just like we got cars instead of artificial horses.
[ Parent | on HN ]
Re:Past Performance is Not Indicative of Future Results (2020) by fossuser on Monday August 02, 2021 @12:19AM (#28032839)
AGI isn't really about artificial humans.

It's about very good general problem solving software that's way beyond the capabilities of humans while not being aligned with human interests. Not because the software is evil, but because aligning values is an unsolved problem (humans aren't even totally aligned - and values also change).

If you have an intelligence that's very good at achieving its goal and you don't have a good way to align its goal with human goals, you can very quickly get into trouble if that intelligence thinks much faster than you do.

https://www.lesswrong.com/posts/mMBTPTjRbsrqbSkZE/sorting-pe...

https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden...

https://www.lesswrong.com/posts/BEtzRE2M5m9YEAQpX/there-s-no...

https://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien...

[ Parent | on HN ]