News for hackers. Stuff that matters.

A comment on Superintelligence–10 Years Later. Parent comment

Re:Superintelligence–10 Years Later by exe34 on Thursday July 04, 2024 @06:37AM (#40873833)
AI is unsafe if it doesn't answer to the board of directors or parliament. Also paperclip maximizers, as opposed to optimizing for gdp.
[ Parent | on HN ]
Re:Superintelligence–10 Years Later by Rhapso on Thursday July 04, 2024 @08:12AM (#40874258)
Yeah, the constant dissonance with AI safety is that every single AI safety problem is already a problem with large corporations not having incentives aligned with the good of people in general. Profit is just another paperclip.
[ Parent | on HN ]
Re:Superintelligence–10 Years Later by ben_w on Thursday July 04, 2024 @11:34AM (#40875713)
Not only but also; they're also every problem with buggy software.

Corporations don't like to kill their own stakeholders; a misplaced minus sign, which has happened at least once*, and your AI is trying as hard as possible to do the exact opposite of one of the things you want.

* https://forum.effectivealtruism.org/posts/5mADSy8tNwtsmT3KG/...

[ Parent | on HN ]
Re:Superintelligence–10 Years Later by digbybk on Thursday July 04, 2024 @12:33PM (#40876177)
Is that dissonance or shows that the concept is generally applicable? Human inventions can be misaligned with human values. The more powerful the invention, the more damage it can do if it is misaligned. The corporation is a powerful invention. Super intelligence is the most powerful invention imaginable.
[ Parent | on HN ]