News for hackers. Stuff that matters.
Re:Superintelligence: The Idea That Eats Smart People by mrfusion on Thursday December 22, 2016 @06:08PM (#13241235)
I've always wondered if the problems an intelligence solves are exponentially hard so even if we build a super intelligence it wouldn't be all that much smarter than we are.

For example compare how many more cities in the traveling salesman problem a super computer can solve vs your grandmas pc. It's more but surprisingly not all that many more.

What do you think of that idea?

Re:Superintelligence: The Idea That Eats Smart People by samatman on Thursday December 22, 2016 @07:16PM (#13241707)
I think the 98% of DNA we share with chimpanzees, with brains almost three times as large, suggests that isn't the case. Language appears to be a force-amplifier in intelligence, and we have no reason to believe that other such do not exist.
Re:Superintelligence: The Idea That Eats Smart People by idlewords on Thursday December 22, 2016 @06:27PM (#13241399)
I think this basic concept of intractability, which programmers are very familiar with, hasn't penetrated far enough into AI world.

Bostrom and Yudkowsky in particular seem happy to hand-wave past computational complexity.

Re:Superintelligence: The Idea That Eats Smart People by Eliezer on Thursday December 22, 2016 @10:41PM (#13242652)
I'm getting kind of sick of people who haven't read 2% of the stuff imagining what they think we've never talked about over the last 16 years.

https://intelligence.org/files/IEM.pdf

Re:Superintelligence: The Idea That Eats Smart People by idlewords on Thursday December 22, 2016 @10:58PM (#13242728)
Reading 2% of your stuff is like reading 400% of my stuff, and I'm verbose as hell.

Edit.

Re:Superintelligence: The Idea That Eats Smart People by ahh on Thursday December 22, 2016 @11:14PM (#13242779)
First off, that's a blatant ad hominem.

Moreover, if your best argument against the guy you're claiming is defrauding everyone is "I can't be bothered to read his work"...

Re:Superintelligence: The Idea That Eats Smart People by idlewords on Thursday December 22, 2016 @11:59PM (#13242950)
I've read Eliezer's stuff, which is what gives my request its special poignancy.
Re:Superintelligence: The Idea That Eats Smart People by devilsavocado on Friday December 23, 2016 @02:07AM (#13243370)
I haven't had time to read the entire thing, so I've skimmed over lots, but there doesn't seem to be much mention of complexity theory in the way I believe idlewords is talking about.

I'm very curious: What happens if the algorithms for general artificial intelligence, and the ability for an AI to improve itself are all NP-Hard problems? Is that covered?

It might be similar to the "intelligence combustion" scenario outlined. But that appears to be a scenario where do not need to be fearing superintelligence.

Re:Superintelligence: The Idea That Eats Smart People by Eliezer on Friday December 23, 2016 @05:18AM (#13244025)
As it says in the paper, the evolution of hominids doesn't look like this; population genetics says that if hominid brains got larger, the marginal fitness returns on neurons went up.
Re:Superintelligence: The Idea That Eats Smart People by Eliezer on Friday December 23, 2016 @05:19AM (#13244032)
Or in a more intuitive sense, humans didn't seem to need brains 1000x as large to get one unit of improvement in practical effectiveness over chimpanzee cognition. But yes, this stuff is complicated, hence the paper not being two paragraphs long.
Re:Superintelligence: The Idea That Eats Smart People by devilsavocado on Friday December 23, 2016 @04:59PM (#13247434)
Thank you, that is a very intuitive explanation.

You've probably read Scott Aaronson's Why Philosophers Should Care About Computational Complexity? [0]. This seems like the perfect area to apply a lot of the questions he brings up. That's what I was looking for as I skimmed through the paper. Maybe that's what idlewords was talking about as well.

[0] http://www.scottaaronson.com/papers/philos.pdf

Re:Superintelligence: The Idea That Eats Smart People by tbt on Thursday December 22, 2016 @09:06PM (#13242221)
Have you seen gwern's essay on computational complexity arguments about AI? https://www.gwern.net/Complexity%20vs%20AI
Re:Superintelligence: The Idea That Eats Smart People by danieltillett on Friday December 23, 2016 @04:42AM (#13243913)
It doesn't even hold with humans. Take a look at John von Neumann and his brain was pushed down the human birth canal.

We really have a poor idea of what super intelligence is - none of us can even understand people much more intelligent than ourselves.