While I agree that we are not currently ready for the arrival of superintelligence, I think it's worth emphasizing that there is time to prepare, that there is useful preparation to be done which may get things to a point where it will be safe (or at least marginally safer), and that if you are feeling worried, it may be best to channel that feeling into useful action and get involved with the research!

Good places to start are the Machine Intelligence Research Institute's Research Guide (https://intelligence.org/research-guide/), which summarizes the aspects of the problem they've worked on and think are important; and the Future of Life Institute's Research Priorities (http://futureoflife.org/static/data/documents/research_prior...), particularly Section 3.

Don't get me wrong, I'm certainly not against trying to prepare against it - perhaps we can discover a breakthrough that does defend against AI (although I'm very doubtful in the long-term). It is worth a serious try though.

I largely agree with you (have been supporting intelligence.org before the topic was cool) but I think if we don't achieve human-friendly AI it's on us; a real failure rather than just succumbing to circumstances (like getting blown away by a supernova). Right now we have control and we can release AGI only if we have very high confidence that it aligns well with human value. The analogy that is somewhat relevant is that basic drives can be controlled even if AI's intellectual power leaves us in the dust, like we know that most normal humans will care about their children (and senile parents) despite being cognitively superior. We have to fight denial (many AI researchers are IMO engaged in it) apathy and fatalism, and in the end with patience we can have something great.

Interesting info came out during interview with Elon Musk recently:

"— He is very, very concerned about AI. I quoted him in my posts on AI saying that he fears that by working to bring about Superintelligent AI (ASI), we’re “summoning the demon,” but I didn’t know how much he thought about the topic. He cited AI safety as one of the three things he thinks about most—the other two being sustainable energy and becoming a multi-planet species, i.e. Tesla and SpaceX. Musk is a smart motherfucker, and he knows a ton about AI, and his sincere concern about this makes me scared."

from: http://waitbutwhy.com/2015/05/elon-musk-the-worlds-raddest-m...

Yes, absolutely there is a reason for concern. I now volunteer for FLI (the organization Musk funded; we deal with both AI and other x-risks, synthetic bio etc). From a somewhat insider view I think there is a lot we can do, and time might be on our side (AGI is a pretty hard problem). I'm cautiously optimistic that we (rational and altruistic humans) can make a difference.

keep up the good fight! Thank you for your efforts