News for hackers. Stuff that matters.
Re:The Last Invention of Man by markan on Friday October 06, 2017 @05:51PM (#15420742)
Yes, this is the problem with AI risk---there's a community pushing hard to gather resources to the cause, but little or no scientific work to be done. This is a rather pathological situation---among other things, the AI risk community makes their own cause look silly, and they promote an unduly negative vision of AGI. I've written more about this here: http://www.basicai.org/blog/ai-risk-2017-08-08.html.

On a positive note, as a piece of science fiction, this was an enjoyable read!

Re:The Last Invention of Man by amasad on Friday October 06, 2017 @09:41PM (#15421762)
I think you sum up their position's fallacy pretty nicely here:

>"if we don't figure out AGI safety now, by the time AGI happens it may be too late"

The keyword for me is "happens". It's as if technology ever happens, or emerges serendipitously. It's like the Kurzweilian exponential law which make it seem as if there is no agency in technology, a natural law. And our role in it is make sure when the aliens or the gods arrive we are prepared for them.

Re:The Last Invention of Man by apsec112 on Friday October 06, 2017 @07:40PM (#15421279)
"but little or no scientific work to be done."

Quite a lot has been written about what scientific work needs to be done. These papers try to summarize possible research directions:

https://arxiv.org/pdf/1606.06565.pdf

https://intelligence.org/files/TechnicalAgenda.pdf

Re:The Last Invention of Man by markan on Friday October 06, 2017 @08:35PM (#15421538)
Yes and no. For safety of narrow AI systems, yeah, there's a lot of scope for research, and that's what your first link gets at.

But for AGI (which is what Tegmark talks about), there's no good way to get a handle on safety yet (other than working towards figuring out AGI).

As for MIRI's agenda, I don't buy that it will help with AGI safety at all. There are a variety of reasons for that, some of which are discussed in the piece I linked above.