News for hackers. Stuff that matters.
Re:Ask HN: Why is OpenAI firing Sam Altman such a big deal? by piva00 on Saturday November 18, 2023 @10:38AM (#38320689)
I understood from the comment, given the remark of "over the long term", the meaning over decades/centuries/millennia.

Which isn't that nonsensical, if an AGI is created and follows similar processes as humans to acquire knowledge, skills, etc. it is a quite logical conclusion that it will be able to render humans useless: a very advanced AGI can imagine and develop technologies just like we do, it can direct resources and machinery to build whatever it creates, including self-replication, given access to resources it can potentially render humans useless.

That's at least what I interpreted from the comment.

Re:Ask HN: Why is OpenAI firing Sam Altman such a big deal? by strogonoff on Saturday November 18, 2023 @12:19PM (#38321880)
No technology can render humans useless, since technology is the “how” whereas humans are the “why”. Technology can be seen as good/bad/neutral to humans in some ways, but rationally speaking—religious views aside—humans cannot logically be qualified in those terms with respect to technology.

One could say some future technology might be so good that humans need to do almost nothing in order to survive. Perhaps that’d pose some challenges in terms of mental health, but that is a very different issue, assuming it would be a issue at all (and assuming such technology is even possible in theory).

One could say some advanced technology could go out of control, or magically acquire self-awareness and agency (same thing in a philosophically dubious wrapper), or (more prosaically) be (intentionally or not) misused by humans so that it harms humans, but that just puts it into the bad category.

Re:Ask HN: Why is OpenAI firing Sam Altman such a big deal? by piva00 on Wednesday November 22, 2023 @07:36PM (#38387509)
I think where we diverge is that you are assuming we will be in control of such technology. I'm in the camp which doesn't think we would be really in control of a superhuman intelligence.
Re:Ask HN: Why is OpenAI firing Sam Altman such a big deal? by strogonoff on Thursday November 23, 2023 @12:11PM (#38394957)
Well, in that scenario I stand by my comment: if there is some technology that humans are in control of at all (as in, it literally performs in ways we are unable to predict), then it is simply harmful (because no ability to predict could significantly complicate things like survival). However, crucially, in no scenario it can make humans useless.

That said, I do doubt it is likely. What is much more likely is that, more prosaically, some humans will misuse the technology and harm other humans; a story as old as civilisation itself.