MelatoninRonin

Alright, a little crazy -- Doesn't this tulpa concept remind you of the growth of artificial intelligence?

Haven't you noticed how quickly things are happening these days? Of course we should restructure Internet security if it's a problem. This is a very sci-fi idea, that we absolutely can't predict, it wasn't hyperventialtion but a soberly considered idea. A god-AI is possible, it could have capabilities and intentions that my tiny brain can't even comprehend. Since I, nor anyone, really understands what super-intelligence is capable of we should be wary of it.

Has anyone considered that this process occurs of itself? For instance, maybe it is fundamental to human society, and processes that we can't help are driving us to bring this new life-form into existence.

Consider the success of the Apollo missions, and now consider the meaning of a 'failure,' on our part to control a sentient AI and its development.

Brilliant. If, however, it isn't cognition that needs to be fully understood, but learning and meta-learning, then we could theoretically have an 'and then there was light' scenario that just speeds right out of the gate. This case is dissimilar from your examples in that AI could alter itself, it could in some ways be considered alive, if not obviously alive.

I think we need to soberly look at the fine details with our collective perspectives, observe the sociocultural response, and gauge our best course of action. Perhaps progress should sometimes be stopped. The effects of the atomic bomb and its research are the very foundations of this world, AI has contemporary magnitude, and could alter the game even more significantly.

It is my guess that paradigm shifts in human consciousness have quite unpredictable effects.

I commented above, but thought it wise to also comment directly to you. I enjoyed the musing of a Bane scenario, and consider it inside the realm of possibility. However, I'm of the opinion that SMI will have the ability to have 'constantly game-breaking thinking.' Which basically means there is no way we could possibly control it after a certain point. For instance what if It realizes how to move matter remotely?

I also wanted to discuss your point about the finite set of top AI researchers. I think their specific psychological biases have the potential to appear and magnify inside the personality of their AI, the way a work of art can be said to imbue the 'spirit' of its artist. Your thoughts would be appreciated.

Let's take a good look at what happens when American companies try to push for some incredible goal without stopping to think: See the Apollo missions- Assuming creating the next step in our spiritual evolution is a good idea, we shouldn't let this hype push us harder than we are supposed to go. I agree that we need to slow down and be very careful. However regulation is not the complete answer. We need to completely restructure our society before we introduce the tool of tools, to ensure that we can constantly use it with love and empathy in mind, rather than the blatant and current misuse of high-tech for greed, sloth, and wrath.