News for hackers. Stuff that matters.

A comment on Superintelligence: The Idea That Eats Smart People. Parent comment

Re:Superintelligence: The Idea That Eats Smart People by ethanbond on Monday May 29, 2023 @06:40PM (#36118960)
This seems quite presumptive. First, intelligence doesn’t seem to be unidimensional. A 140 IQ person can be fooled by an optical illusion just the same as anyone else. It’s just not a problem that’s able to be intelligenced away from our cognition. That doesn’t mean a 140 IQ person can’t beat an 80 IQ person in many many other competitions of intelligence.

Second, if you are truly “accepting the premise” of superintelligence, a superintelligence would know exactly this line of reasoning and just opt to at least mimic vulnerability to prompt injection.

I wouldn’t hang civilization on this proofpoint. Doesn’t seem meaningful at all.

[ Parent | on HN ]
Re:Superintelligence: The Idea That Eats Smart People by usrbinbash on Tuesday May 30, 2023 @06:26AM (#36123211)
> I wouldn’t hang civilization on this proofpoint.

And I wouldn't risk, or slow down getting to, the potential benefits of developing AI further, which are tangible and measurable, because of vague threat scenarios with little to no evidence or methods of measurement that seem like good plots for a SciFi B-Movie.

Pascals wager, as an argument, relies in no small part on the assumption that to believe in a god-like entity doesn't come at a significant cost. Slowing down or abandoning the development of AI however, does.

[ Parent | on HN ]
Re:Superintelligence: The Idea That Eats Smart People by ethanbond on Tuesday May 30, 2023 @06:45AM (#36123314)
“I don’t know how to mitigate the risk without giving up something I like” is a completely different argument than “there isn’t risk.”

AI optimists tend to conflate the two for some reason.

[ Parent | on HN ]