News for hackers. Stuff that matters.
Re:Be a Visiting Fellow at the Singularity Institute by indrax on Thursday May 20, 2010 @12:56AM (#1363283)
Note that Shaping the Singularity was taken as the same task as averting human extinction. The talk of AI is secondary, it comes up because many of us think that AI can offer good solutions. In looking at AI, it also seems apparent that AI is itself an existential threat.

Is shaping the singularity the same as averting human extinction? If the singularity itself poses existential risks (this seems obvious,) and if extinction is inevitable without a singularity (this seems likely,) then we need a singularity that is 'safe and soon'. (though the initial wording does seem to gloss over the need to not go extinct in modern ways.)

Put another way: We can do the most work with the most powerful tools. There is a great need to use very powerful tools correctly. If we can even maybe influence the work done by highly powerful mid-singularity tools, we can accomplish far more 'correct work' than we could just using our modern tools. (Again, this does require that we survive long enough for the singularity to happen, and that you don't have a strong discount for future vs. present utility.)

Personally, I think creating a super-tool which can only be used correctly is the best approach.

Re:Be a Visiting Fellow at the Singularity Institute by thunk on Thursday May 20, 2010 @01:59AM (#1363356)
> Is shaping the singularity the same as averting human extinction? If the singularity itself poses existential risks (this seems obvious,) and if extinction is inevitable without a singularity (this seems likely,) ...

The answer is trivially "no". It's easy to imagine a future in which we continue to exist and the Singularity never arrives. Perhaps the rate of progress will continue to increase at a decreasing rate. Perhaps it will go linear, or even sigmoid. Perhaps we will achieve a Kardashev Type I civilization without ever going properly singular. It's not a given and perhaps not even likely that singularity is synonymous with the survival of our species.

We have the existence proof for general intelligence embodied in about ~3 lbs of protein. No more, no less. Being certain of the inevitability of the Singularity is closer to religious than skeptical thinking.

Re:Be a Visiting Fellow at the Singularity Institute by rms on Thursday May 20, 2010 @03:35AM (#1363458)
On Less Wrong and at the SIAI house, we quickly learn to stop saying certain or "100%". Do you think a Singularity within the next 50 years is 95% likely? 80% likely?

Also, for certain definitions of inevitable, extinction is inevitable without a Singularity. A Type 1 Civilization could at absolute best colonize a few surrounding solar systems, but eventually the stars will burn out. Though a universe where we hit Type 1 but never Type II would be rather interesting.

Re:Be a Visiting Fellow at the Singularity Institute by pook on Thursday May 20, 2010 @03:41AM (#1363469)
You and Clippy should put up an Ask HN AMA post.

Though I suspect you'll have to lure Clippy here with the promise of persuading angel investors into funding nanotechnological paperclip foundries.

Re:Be a Visiting Fellow at the Singularity Institute by indrax on Thursday May 20, 2010 @04:18AM (#1363508)
I did not say that the Singularity was certain or inevitable. I do kind of take it for granted but if you're saying it's not going to happen, that's a whole different argument.

I'm not saying "If humans continue to exist, we will certainly do the singularity thing." I'm saying "We are likely going to go extinct unless things change in a really really big way." And of course those big changes also have to not kill us.

Re:Be a Visiting Fellow at the Singularity Institute by derefr on Thursday May 20, 2010 @02:02AM (#1363358)
> extinction is inevitable without a singularity (this seems likely,)

Can you explain this point further? If we can manage to, say, genetically engineer ourselves into healthy immortals that are otherwise pretty much as we are today, invent FTL drives to spread ourselves across the universe, and discover a way to reverse entropy to avoid the big crunch—and after doing all that we still haven't turned ourselves into para-omniscient universe-scale entities—then do we really need any sort of technological singularity, or can we just decide to hold off on it indefinitely?

Re:Be a Visiting Fellow at the Singularity Institute by indrax on Thursday May 20, 2010 @03:54AM (#1363484)
Even just decent sublight space colonization would probably be enough to make human survival fairly reliable for the foreseeable future, but I don't think ANY space colonization is likely in the foreseeable future. On that count, I think we're stuck on this rock for a while (unless the singularity makes it easy) and will sooner or later screw up our rock. (war, probably)

There probably are ways we could steer into a stable human populated reasonably pleasant universe, but we're in a car filled with guns liquor and dynamite on a winding mountain road in the dark.