It depends on what you mean by self-consciousness. On the one hand it can be processes for understanding oneself and modifying oneself to better survive in the environment, on the other you have the illusory "qualia" / subjective experience. It seems possible to me to have a self-improving process and an active process to evaluate the world state and self state without consciousness, and that this could result in something life-like. In this sense, is AI far away? It doesn't have to be necessarily.

It doesn't make sense to me personally why we need subjective experience, nor what it is in a physical sense. We can have agency and variability of choice without it, but at the same time we know how powerful it is in humans. Until we know exactly how it functions, i don't think anyone can claim with any certainty if it will happen, or how far away it is.

Agreed. As someone who works in AI (and knows its limitations) it's frustrating to see the layperson getting nervous about AI after watching Chappie and hearing some of these quotes from Gates and the like. Human level AI is way way way out.

I think it's reasonable for the layperson to get a bit nervous. Even if it's 50 or 200 years, advanced, human-brain-like-or-better AI will happen.

Imagine the public sentiment if mass media made people aware of the inevitability of the nuclear bomb 50 to 200 years in advance and people (not just the government) were actively working on developing it.

In the analogy of nuclear bomb, it's more likely than not a public awareness of nuclear power would have prevented nuclear plant (the good thing), nuclear bomb (the bad thing) is extremely unlikely to not be developed anyway.

Maybe, it's hard to know for sure, just like it's hard to know now :)

I think that had nuclear physics been more obvious 50-200 years earlier, it may have lead to a lot more practical private-sector development. In the case of power generation, this is seriously beneficial.

How will private-sector AI turn out? Will they boot up sentient AIs in Docker, then discard them? Is that OK? Who even knows?

Seems like it's a popular sentiment to think of Elon Musk as Tony Stark ;). I believe you mean to say "Gates, Hawking and Musk". Stark doesn't tend to talk about AI, he build them.

Was just a little joke ;)

Did you just include Tony Stark in a list of people worried about AI? ;)

Tony Stark, Elon Musk ... Same person, different multi-verse.

Hawking always had his eyes on the stars, perhaps he isnt qualified to comment on AI?

I couldn't agree more. 100 or 200 years seems feasible.

I think you mean, simply, conscious or self-aware. A self-conscious AI would spend too much time worrying about what the other robots think of it.