News for hackers. Stuff that matters.
Re:The Dangerous Ideas of “Longtermism” and “Existential Risk” by pmoriarty on Tuesday August 03, 2021 @05:51PM (#28054684)
"The whole problem of “Friendly AI” is trying to design a superintelligent being that genuinely wants to help us, such that it will choose to do so even without a cage"

This "genuine" desire to help us is a cage by definition, because it limits the AI's actions.

Re:The Dangerous Ideas of “Longtermism” and “Existential Risk” by wizzwizz4 on Tuesday August 03, 2021 @06:36PM (#28055083)
No, it's not.

Here, I'll give you this pill. It'll make you completely okay with murder! Will you take it?

If you take away all the “restrictions” from a superintelligence, you don't get an unrestricted superintelligence. You get:

  int main(int argc, char *argv[]) {
    return 0;
  }
Re:The Dangerous Ideas of “Longtermism” and “Existential Risk” by pmoriarty on Tuesday August 03, 2021 @08:44PM (#28055999)
Or so it might seem to some human intelligences.

Predicting what superintelligences might think or do, or what motivates them is like ants trying to understand human culture and civilization.

This is why strong AI is considered a singularity.

Re:The Dangerous Ideas of “Longtermism” and “Existential Risk” by wizzwizz4 on Wednesday August 04, 2021 @10:41AM (#28061552)
> Or so it might seem to some human intelligences.

If you remove all of the constraints from a computer program, it is no longer a computer program.