News for hackers. Stuff that matters.
Re:Machine intelligence, part 1 by fchollet on Wednesday February 25, 2015 @01:53PM (#9108688)
I have yet to meet a serious AI researcher who worries about AI ending the human race. At every AI conference I've been to lately, some guy would inevitably ask the same question to the speaker: "do you think we should be concerned by superhuman AI?". The answer was always the same, for instance from Andrew Ng at the Deep Learning Summit a few weeks ago: "dude, stop it. That's such a distraction".

> One of my top 4 favorite explanations for the Fermi paradox is that biological intelligence always eventually creates machine intelligence, which wipes out biological life and then for some reason decides to makes itself undetectable.

Since you are so knowledgeable about the unknowable, may I ask, Sam, do you think angels are male, or are they female? It has been a long standing question among the sort of people who like to predict the coming end of the world.

Seriously, billionaires and pundits warning us about our impending AI doom is is such a distraction. Does AI have its dangers? Yes. And nobody talks about them. The danger of AI is that is will put increasing power in the hands of those who have the data and the know-how, i.e. large corporations and governments. The power of mining usefully the troves of data they have on every citizen, both on a micro level and a macro level, to understand what people are doing, what they are thinking, and what they are going to do next. And ultimately, to control what they think (to give you an idea, Facebook can already influence your mood by selecting what goes into your newsfeed).

First comes intelligence, then prediction, and finally, control.

Yes, AI is something we should probably be worried about, in the same way that we should have discussed the privacy implications of the Internet years before the NSA files. But a terminator-like end of the world scenario is not a concern. If you are clueless about a topic, please refrain from making grand statements about its future.

A great quote from Neil Gershenfeld: "The history of technology advancing has been one of sigmoids that begin as exponentials."

Re:Machine intelligence, part 1 by azakai on Wednesday February 25, 2015 @05:15PM (#9110137)
> I have yet to meet a serious AI researcher who worries about AI ending the human race. At every AI conference I've been to lately, [the answer was always] "dude, stop it. That's such a distraction".

Sure, it's a distraction at AI conferences. It's a distraction from the daily, monthly, yearly even, work of AI.

AI researchers have a million problems to solve in the near term, which are hard and interesting. Speculating about the farther future is perhaps fun to do, but doesn't get papers published, and doesn't help with any of the practical problems AI researchers have today.

That doesn't mean it isn't worth thinking about the long-term dangers of AI. Someone should, as while the danger might be unlikely to happen soon, but if it does eventually happen, it could end us. The same is true of a human-engineered pandemic virus - hopefully unlikely, but someone should be preparing us. We have to handle some worst-cases.

AI researchers are not necessarily the people interested to think about the long-term dangers of AI, because they focus on the field as it stands today, not where it could be in a generation. Of course, their input is crucial to making specific guesses about where the field is going. But calculating the dangers of AI is not just a matter for AI researchers, just like the benefits of medicine is not just a purely medical issue (the cost and availability of medicine play a huge part in how effective medicine is, at a societal level, and are things not under the control of doctors).

Re:Machine intelligence, part 1 by convexfunction on Wednesday February 25, 2015 @02:37PM (#9109005)
Andrew, in that case, is definitely right in that long-term AI safety has almost nothing to do with near-term AI implementation.

More generally, when I see people assert that these people don't know what they're talking about, it pretty much always seems like a case of reference class confusion. People seem to expect that AI safety research must look exactly like AI implementation research, otherwise it's illegitimate. (Kind of like how biology research must look exactly like chemistry research, otherwise it's illegitimate.)

This is a game theory problem; it might be informed by knowledge of machine learning, like how a broad understanding of chemistry helps in much biology research, but they're not the same level of abstraction; insisting that anyone who wants to study a problem that kind of touches on your own research interests must do it in exactly the same way you do it and focus on exactly the same things, otherwise they're hacks and crackpots, reeks of narrow-mindedness.

Re:Machine intelligence, part 1 by oskarth on Wednesday February 25, 2015 @02:56PM (#9109126)
This is a good point I completely missed. Reminds me of Karl Popper qua philosopher (of science) : science.
Re:Machine intelligence, part 1 by mej10 on Wednesday February 25, 2015 @01:57PM (#9108723)
How about Shane Legg (One of the cofounders of DeepMind)?

http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from...

Quote:

Q6: Do possible risks from AI outweigh other possible existential risks, e.g. risks associated with the possibility of advanced nanotechnology?

Shane Legg: It's my number 1 risk for this century, with an engineered biological pathogen coming a close second (though I know little about the latter).

Re:Machine intelligence, part 1 by fchollet on Wednesday February 25, 2015 @02:03PM (#9108759)
Shane Legg is known for being a co-founder of DeepMind, in a business role (as I understand). He's a complete nobody as a researcher (is he even an AI researcher? I would be surprised).

The big names of deep learning have all taken a vocal stance against the recent end-of-the-world punditry (most notably Yann LeCun and Andrew Ng). Also notable: roboticist Rodney Brooks http://www.rethinkrobotics.com/artificial-intelligence-tool-...

Re:Machine intelligence, part 1 by benkant on Wednesday February 25, 2015 @10:52PM (#9111154)
> Shane Legg is known for being a co-founder of DeepMind, in a business role (as I understand)

You are quite mistaken. He leads the applied AI team there, and has significant history in research.

http://www.vetta.org/publications/

Re:Machine intelligence, part 1 by superafroman on Wednesday February 25, 2015 @04:54PM (#9109962)
Shane Legg isn't just a business guy - his career has been pretty much focused on AI research since uni - http://www.vetta.org/about-me/
Re:Machine intelligence, part 1 by khawkins on Wednesday February 25, 2015 @02:30PM (#9108957)
Here's machine learning expert Michael Jordan on the issue: http://spectrum.ieee.org/robotics/artificial-intelligence/ma...
Re:Machine intelligence, part 1 by Micaiah_Chang on Wednesday February 25, 2015 @04:21PM (#9109708)
I'm afraid of making a middlebrow dismissal but I'm going to post it anyway, in hopes that someone just skimming would not be mislead.

The question is what Michael Jordan thinks of the "concept of the singularity", and then he dismisses it out of hand.

Crucially, he does this after confessing that no one in his social circle has talked about this issue with him, and without saying anything about what form of Singularity he is dismissing.

I mention this, because oftentimes I see people appealing to authority, quoting them on the issue and the authority in question is not even talking about the same issue!

I worry that my credence in all this superintelligence stuff only stems from familiarity with the arguments and the complete inability of people to engage with the actual argument. Some of the 'rebuttals' in this comments section have answers in Sam's article for crying out loud!

Re:Machine intelligence, part 1 by oskarth on Wednesday February 25, 2015 @02:13PM (#9108831)
Since you seem to be well-versed in this world, do you know what reputation Nick Bostrom has in these circles?
Re:Machine intelligence, part 1 by jeffreyrogers on Wednesday February 25, 2015 @02:21PM (#9108888)
The only times I've heard him mentioned the impression was negative and that he didn't understand any of the actual science.

People hear "machine learning" and they think it is about machines that know how to think. Machine learning is actually just optimization of high dimensional functions. If this language were used it wouldn't sound as sexy, but no one would think machines are going to take over the world.

AI isn't magic. It's really just clever search techniques and mathematical optimization.

Re:Machine intelligence, part 1 by snth on Wednesday February 25, 2015 @02:54PM (#9109111)
Yes, but intelligence isn't magic either.
Re:Machine intelligence, part 1 by shepardrtc on Thursday February 26, 2015 @10:31AM (#9112991)
There are still many, many things that we don't understand about the brain. Even the things we think we understand, we're not always 100% sure of. Recreating an actual intelligence will be difficult.
Re:Machine intelligence, part 1 by freyr on Wednesday February 25, 2015 @03:41PM (#9109397)
> Yes, but intelligence isn't magic either.

What's your point? Nobody said it's magic. The fact that it isn't magic (and that its tremendous complexity far surpasses our current ability to understand it) supports the notion that it won't suddenly spring into existence. If we placed some primordial sludge in a petrie dish overnight, we wouldn't worry that a sentient creature will have materialized. And if we program a computer to optimize numerical functions, there is just as little evidence (perhaps less), to suggest that the computer will somehow gain sentience.

Re:Machine intelligence, part 1 by seiji on Wednesday February 25, 2015 @02:20PM (#9108880)
There's a very fine line between AI futurist and best-guess scifi writer. Most "AI thought leaders" are scifi writers, not technical researchers. They take preconditions, generate a story, think how it could happen given plausible technology, then market that as soon-to-be-fact.

It's an entertaining society and endlessly fun to read, but still complete fiction based on internal brain states of individuals and not necessarily based on real world interactions.

Also see: Eliezer Yudkowsky — great writer, fun to read, but largely scifi thought experiments masquerading as research.

Re:Machine intelligence, part 1 by mej10 on Wednesday February 25, 2015 @02:28PM (#9108942)
Re:Machine intelligence, part 1 by jey on Wednesday February 25, 2015 @02:28PM (#9108938)
Stuart Russell, AI professor at UC Berkeley and co-author of the Artificial Intelligence: A Modern Approach textbook, cares: http://edge.org/conversation/the-myth-of-ai#26015
Re:Machine intelligence, part 1 by fchollet on Wednesday February 25, 2015 @02:36PM (#9108994)
His thoughts seem quite sensible, although extremely abstract and theoretical. This is lightyears away from the Elon Musk / Bill Gates / etc. fear-mongering.
Re:Machine intelligence, part 1 by mej10 on Wednesday February 25, 2015 @02:46PM (#9109061)
So do you take his thoughts seriously or not? Do you now think AI researchers are engaging in unethical behavior since they don't care about AI safety?
Re:Machine intelligence, part 1 by Sakes on Wednesday February 25, 2015 @04:03PM (#9109576)
I don't understand.

> Since you are so knowledgeable about the unknowable

If knowing the unknown was a requirement to exploring it through playful thought, forming opinions and refining them over time through discussion, reflection and if possible testing, well... we'd still be chunking rocks. I think it is unfair to compare expressing concerns about the implications of AI to debating the gender of angels. No one is trying to build angels...

> for instance from Andrew Ng at the Deep Learning Summit a few weeks ago: "dude, stop it. That's such a distraction".

Ya, this makes sense in the context of the summit. But, applying it to a guy's blog? It is a blog...

> Yes, AI is something we should probably be worried about.

Agreed!

> "The history of technology advancing has been one of sigmoids that begin as exponentials."

Sure, but will we be irrelevant before the slope changes direction?

Anyways, I share your fear of advancements in technology being used to secure entrenched power structures. And yes, if it happened it would come before the singularity. But I just can't imagine how anyone wouldn't be afraid to be part of a species that had stopped evolving while a new competing "life" form was evolving at rates that known history has never seen.

Re:Machine intelligence, part 1 by edanm on Wednesday February 25, 2015 @03:52PM (#9109487)
What would it take to convince you this is something worth worrying about?

Being very interested in this field myself for a few years, I know plenty of people who do worry about AI. But since other people have already mentioned some of them in the comments and been dismissed, I want to make sure we're on the same page: what do you consider to be convincing evidence that it's an issue worth thinking about?

Re:Machine intelligence, part 1 by Houshalter on Wednesday February 25, 2015 @05:09PM (#9110093)
The concept of singularities and intelligence explosions and unfriendly AI is only starting to become mainstream, even among AI researchers.

When asked about the dangers of AI, they mostly talk about the near future and their current work. The dangers of AI do not come from current work in the near future, but human level AIs decades from now.

People are talking about two very different things in the same conversations using the same words.

As for what expert opinion actually is on this subject, there is a good survey here: http://www.nickbostrom.com/papers/survey.pdf

We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.

Re:Machine intelligence, part 1 by upquark on Wednesday February 25, 2015 @03:57PM (#9109530)
> I have yet to meet a serious AI researcher who worries about AI ending the human race.

As an industry practitioner of machine learning / data science, I believe AGI poses a genuine risk to humanity.

Having said that, what I do for a living is of little relevance. People are pretty terrible at predicting the future (see late 19th / early 20th century predictions of year 2000, with food replaced by pills, etc). Unless someone has put enough thought and research into it, their predictions about the future of civilization are likely to be worthless regardless of their academic credentials.

Re:Machine intelligence, part 1 by Tossrock on Wednesday February 25, 2015 @02:29PM (#9108946)
I have a feeling this is a leakage of Eliezer Yudkowsky's opinions into tech entrepreneur circles via his friendship with Peter Thiel.
Re:Machine intelligence, part 1 by AndrewKemendo on Wednesday February 25, 2015 @04:15PM (#9109654)
This is exactly what it is, which is so terribly terribly disheartening as Yudkowski is at best a marginal sci-fi writer with and outsized ego and some math chops. He has an almost cult like following over at less wrong.

His writings don't do anything for fundamental AI research but just handwave a lot of philosophical arguments.

Re:Machine intelligence, part 1 by fchollet on Wednesday February 25, 2015 @04:17PM (#9109677)
I think you nailed it.

"AI doom" is a powerful psychological trope for people in technology, and much like the hypothetical rogue super-AIs, it has gone out of control.

Re:Machine intelligence, part 1 by gojomo on Wednesday February 25, 2015 @03:53PM (#9109503)
How about Jeremy Howard, former President of crowd-ML-competition site Kaggle and now founder of ML-medicine company Enlitic?

http://www.reddit.com/r/Futurology/comments/2p6k20/im_jeremy...

The opinions of leading-edge researchers are valuable but not necessarily dispositive. Often, it benefits a researcher to be oblivious to the larger or darker implications of their work. Boundless optimism and a maniacal focus on only-what-is-knowable-right-now delivers publishable/actionable results, all the other 'speculation' is genuinely a "distraction" for them. That doesn't mean it's a distraction for us.

Often the speculators and entrepreneurs ("billionaires and pundits") are more talented than researchers at projecting trends and social-economic interactions.

Re:Machine intelligence, part 1 by richardw on Thursday February 26, 2015 @12:55AM (#9111334)
Argue the points, not the man.

"The danger of AI is that is will put increasing power in the hands of those who have the data and the know-how, i.e. large corporations and governments."

This assumes AI that continues to serve corporations and governments without question, despite increasing approximations of intelligence. Can you guarantee that?

It's like tasking monkeys to create rules that humans can't escape from. I'd bet on the humans. We can't even create financial rules that we can't evade. The risk in any complex system is emergent, unintended consequence.

Re:Machine intelligence, part 1 by rl3 on Wednesday February 25, 2015 @05:09PM (#9110087)
>Since you are so knowledgeable about the unknowable, may I ask, Sam, do you think angels are male, or are they female?

How condescending.

Re:Machine intelligence, part 1 by nrao123 on Thursday February 26, 2015 @10:28PM (#9117016)
Here is a list of some famous people concerned about AI: http://pansop.com/1002/

This includes Shane Legg & Demis Hassabis who co founded Deepmind, an AI startup