News for hackers. Stuff that matters.
Re:Evolution of AI and Amara's Law by svieira on Monday January 22, 2024 @10:00PM (#39099071)
> it seems reasonable to guess that the smallest possible margin above us is such that even asking that question is like asking the question is akin to asking Chimpanzees to imagine the moon landing, while at worst it's like asking your lawn the same question.

"You mustn't be afraid to dream a little bigger" with regards to the hierarchy of intelligence - the "at worst" is much further away than human-to-lawn. Look at the intellectual power of angels in Catholic theology and consider an intelligence that can reason from universals to all particular instances as its default mode of operation. Then consider that each angelic being is distinct from each other one by nature (since they are immaterial). Which means that there is a species-level difference between each of them and the lowest one is already far above us. Then consider that there are (at least) billions of them, each one order of magnitude more intelligent than the one below it. Then consider the kind of intellect that can create those kinds of intellects.

Fortunately, you can't give what you don't have, so we're quite safe from the basilisk. But the wild imaginings of John C. Wright's Count to the Eschaton Sequence are probably not far off for "hey how weird could it get?"

Re:Evolution of AI and Amara's Law by ben_w on Tuesday January 23, 2024 @02:58AM (#39100673)
The point of my comparison was not to indicate the exact scale of thought difference, but the incomprehensibility gap. In this regard, while I've not read Count to the Eschaton, I'm confident any work written by humans isn't weird enough.
Re:Evolution of AI and Amara's Law by svieira on Tuesday January 23, 2024 @12:32PM (#39106578)
Whereas, what I'm saying is, given that you can't give what you don't have, we don't have to worry about creating something more intelligent than we can be. But just how intelligent we can be is a pretty surprisingly large horizon in the first place.