lukeprog

I know for a fact that at least 7 of the top AI people that I listed above think intelligence explosion / superintelligence is plausible and that it's not at all clear that will go well for humans. I'm pretty sure all of them think AGI will be feasible one day and will be fairly hard to make reliably safe, at the very least to the degree that current safety-critical systems require a bunch of extra work to make safe (beyond what you'd do if your AI system has no safety implications, e.g. detecting cats on YouTube).

I can't say which people believe which things, because most of them communicated with me in confidence, except for Stuart Russell and Shane Legg because they've made public comments about it about their worries about intelligence explosion:

http://futureoflife.org/PDF/stuart_russell.pdf

http://lesswrong.com/lw/691/qa_with_shane_legg_on_risks_from...

> On the other hand you have (essentially) the entire mainstream academic AI/ML community, who are mainly concerned that the hype is so intense it might lead to another AI winter

This doesn't seem true. E.g. the signatories on the Future of Life Institute's recent open letter about AI progress and safety include:

* Stuart Russell & Peter Norvig, co-authors of the #1 AI textbook

* Tom Dietterich, AAAI President

* Eric Horvitz, past AAAI President

* Bart Selman, co-chair of AAAI presidential panel on long-term AI futures

* Francesca Rossi, IJCAI President

* All founders of Deep Mind and Vicarious, two leading AI companies

* Yann LeCun, head of Facebook AI

* Geoffrey Hinton, top academic ML researcher

* Yoshua Bengio, top academic ML researcher

...and many more.

Goldman Sachs did not donate to the charity that plausibly saves lives at a price of $2500 (Against Malaria Foundation). If I had to guess, they gave it to groups who don't even try to save lives, or to groups who save lives at a cost 100 times or more greater than $2.5k/life.

Or really, that entire series of posts, called 'Raw Nerve': http://www.aaronsw.com/weblog/rawnerve

One of the best I've read, too, and I've read hundreds. I also highly recommend '59 seconds' by experimental psychologist Richard Wiseman.

Paul Graham is one of my favorite writers. He's engaging, surprising, simple, and brief. And he writes on topics of interest to everyone. Does anyone else know good essayists that share these same qualities?

I rather like John T. Reed as well: http://johntreed.com/headline.html

Epic. In my dreams, this article will become so popular that political commentators will use your Disagreement Hierarchy to evaluate political speeches. (Most political debate rarely rises above DH4.)

I'm going to reference your article every time I respond to criticism. Thank you.

Love your other stuff, too, Paul.