News for hackers. Stuff that matters.
Re:DeepMind to Work Directly With Google’s Search Team by nl on Monday January 27, 2014 @11:24PM (#7135296)
I think most science degrees have some units relating to ethics. I know my computer science degree had something called something like "computing in society" that kind of tackled it.

But the questions regarding - say - what to do with predictions from a super-intelligent sentient AI weren't really addressed.

I think one problem is that anyone attempting to address those questions in a serious fashion risks being branded a crackpot.

Re:DeepMind to Work Directly With Google’s Search Team by endtime on Tuesday January 28, 2014 @01:17AM (#7135665)
The MIRI folks (intelligence.org) are trying to address exactly that, though I'm sure some consider them crackpots.
Re:DeepMind to Work Directly With Google’s Search Team by nl on Tuesday January 28, 2014 @01:26AM (#7135678)
Yes, they are the only group I'm aware of that is addressing this.

I'd note that they publicise their ethics work a lot less then they used to (eg, nothing on the http://intelligence.org/research/ page).

http://intelligence.org/files/MachineEthicsSuperintelligence... is good though.

Re:DeepMind to Work Directly With Google’s Search Team by eli_gottlieb on Tuesday January 28, 2014 @04:03AM (#7136128)
Their former incarnation as the Singularity Institute were crackpots. Luke Muehlhauser has done a hell of a lot to make them a more serious organization.

The Future of Humanity Institute at Oxford University also work on this stuff, with plenty of mainstream credibility.

Re:DeepMind to Work Directly With Google’s Search Team by endtime on Tuesday January 28, 2014 @06:13PM (#7140526)
EY is still just as involved as he was before, no? Vassar having left probably helps with the crackpot perception (I do not personally have an opinion on the matter, I just know that that's the impression he gives many folks). But given that you used the plural "crackpots", I'm wondering if perhaps there were other folks you think MIRI is better off without? No need to name names, just curious.

(I agree about Luke, he's a great public face.)

Re:DeepMind to Work Directly With Google’s Search Team by eli_gottlieb on Wednesday January 29, 2014 @05:18AM (#7143046)
Generally, I would say they crossed the line from Crackpot to Legitimate when they switched from:

"We are the only people on this planet who have ANY understanding of THE MOST IMPORTANT PROBLEM EVER and the only way you puny humans can catch up is to indoctrinate yourselves by archive-diving our blog, and the only way to SAVE THE WORLD is to send us all your money before the monster can eat you."

to the new message, which is:

"This is a serious problem which could have dire consequences. We believe we have a plan for dealing with it, but have stopped claiming to be the only people on the planet who can deal with it. We are going to be publishing various portions of our plan as peer-reviewed papers and broad discussions with the informed public of mathematicians, which means we are admitting to being mere mortals who in fact have peers. We are also collaborating with this other organization, who are part of one of the world's premier universities. While we still take donations, we're going to stop demanding you send us all your money."

Overall, I am still waiting for them to get even less cultish before I'm willing to donate any more than the price of a Harry Potter novel, but I still find LessWrongians to be one of the most downright fun bunches of nerds to hang with on the entire internet.

Re:DeepMind to Work Directly With Google’s Search Team by endtime on Friday January 31, 2014 @06:19PM (#7159848)
Yeah, I used to hang out at the NYC LW meetup a lot, it was a lot of fun.

I agree that the second message is better in pretty much every way, but I'm not sure they're mutually exclusive in that I think they fall on different parts of the "what we say publicly <--> what we believe" spectrum. Though I'd agree that they seem to have moderated a bit.