We all think of incredibly futuristic sentient robots, but what we should really be worried about are the implications this acquisition will have on Google Search, a tool that is already used by millions of people. A tool that millions of people already superimpose to "the Internet" itself. A tool that already has a strong and unmitigated influence on people choices.
That may not be completely clear in the U.S. Market, but take a stroll in peripheral markets and, to quip the famous commercial, you'll see why 2014 already looks a lot like "1984".
But the questions regarding - say - what to do with predictions from a super-intelligent sentient AI weren't really addressed.
I think one problem is that anyone attempting to address those questions in a serious fashion risks being branded a crackpot.
I'd note that they publicise their ethics work a lot less then they used to (eg, nothing on the http://intelligence.org/research/ page).
http://intelligence.org/files/MachineEthicsSuperintelligence... is good though.
The Future of Humanity Institute at Oxford University also work on this stuff, with plenty of mainstream credibility.
(I agree about Luke, he's a great public face.)
"We are the only people on this planet who have ANY understanding of THE MOST IMPORTANT PROBLEM EVER and the only way you puny humans can catch up is to indoctrinate yourselves by archive-diving our blog, and the only way to SAVE THE WORLD is to send us all your money before the monster can eat you."
to the new message, which is:
"This is a serious problem which could have dire consequences. We believe we have a plan for dealing with it, but have stopped claiming to be the only people on the planet who can deal with it. We are going to be publishing various portions of our plan as peer-reviewed papers and broad discussions with the informed public of mathematicians, which means we are admitting to being mere mortals who in fact have peers. We are also collaborating with this other organization, who are part of one of the world's premier universities. While we still take donations, we're going to stop demanding you send us all your money."
Overall, I am still waiting for them to get even less cultish before I'm willing to donate any more than the price of a Harry Potter novel, but I still find LessWrongians to be one of the most downright fun bunches of nerds to hang with on the entire internet.
I agree that the second message is better in pretty much every way, but I'm not sure they're mutually exclusive in that I think they fall on different parts of the "what we say publicly <--> what we believe" spectrum. Though I'd agree that they seem to have moderated a bit.
Then it ends up taking days.
Nevertheless, I think predictions like this from people actively working in the field are something that shouldn't be dismissed out of hand.
I know 10 years ago I didn't think I'd see self-driving cars in my lifetime.
He then started accomplishing it within three, if the "Playing Atari via Deep Learning" paper is to be believed.
Also, I thought this was hilarious:
In a 2011 interview that predated DeepMind, co-founder Shane Legg said he gave only a 50 percent chance that human level-machine intelligence would exist by 2028.
ONLY?!? By 2028? That is a pretty radical prediction. Indeed, Legg says there is a 10% chance of human level intelligence by 2018, and 90% by 2050[1].
Legg has multiple papers published in the field of measuring machine intelligence, so I guess he has a pretty good view of the field.
If that 2028 prediction is even close to correct then I think setting up an ethics board now probably is the correct thing to do. (the notion that DeepMind asked that Google create an internal ethics board as a condition of the acquisition, as reported by The Information, had some AI researchers griping)
Edit: Legg's posting on his blog in 2011 deserves to be read:
I’ve decided to once again leave my prediction for when human level AGI will arrive unchanged. That is, I give it a log-normal distribution with a mean of 2028 and a mode of 2025, under the assumption that nothing crazy happens like a nuclear war. I’d also like to add to this prediction that I expect to see an impressive proto-AGI within the next 8 years. By this I mean a system with basic vision, basic sound processing, basic movement control, and basic language abilities, with all of these things being essentially learnt rather than preprogrammed. It will also be able to solve a range of simple problems, including novel ones.
[1] http://lesswrong.com/r/discussion/lw/691/shane_legg_on_risks...
[2] http://www.vetta.org/2011/12/goodbye-2011-hello-2012/