We already have true AI - it's a vast, thriving industry.

Or how about calling that vast, thriving industry "weak AI," or "clever algorithms," which is what they really are. The original definition of AI was what we now call strong AI, but after some lesser problems were solved without actually creating strong AI, we had to come up with some name for those.

I want to see an AI that can improve itself by developing new algorithms for arbitrary tasks. I wonder how far off we are from that now?

You know, if you're at the point where you can give a human-readable spec of the problem and the AI can make a passable attempt at it, that's basically the Turing Test -- hence why I think it deserves its status as holy grail. Something that passes would really give the impression of "there's a ghost inside here".

Rather than a ghost, I wonder if we'll ever have the average person looking at brains and thinking "there's a program inside here."

And then to reverse it, imagine that the world really is some kind of massive simulation... and that there are backups of the save()-ed :)

The problem is that fundamentally all our AI techniques are heavily data-driven. It's not clear what sort of data to feed in to represent good/bad algorithm design

Interestingly just ~5 years ago the team "AI" was frowned upon and people insisted in using the term "machine learning". The reasoning was that "AI" is just too convoluted a term because people will insist on comparing with humans and then whole question of consciousness invariably arises which renders scientific inquiry into undesirable debates.

I think that the refusal to seriously engage with consciousness is the main obstacle to progress toward general AI.

I guess it boils down to a difference in opinion as to what it means to be intelligent.

IMO, AlphaGo isn't "intelligent" because all it can do is play Go. For example, it can't be taught to play chess without completely reprogramming it.

Surely it's using a lot of clever algorithms, but where's the intelligence?

TBH, there's not much point to this post, because the intelligence in AI has been twisted more towards your usage almost since the beginning.

I agree with you and would like to double down: Intelligence is defined as "...the ability to acquire and apply knowledge and skills."

Even if Alpha-Go could play chess, manage air traffic control, and play GO the same time, it would never know it was doing that. As you mentioned above, Alpha-Go's "intelligence"is specifically pre-programed algorithms and accurate numeric inputs. If it can't create it's own algorithms or translate abstract data into numbers it can crunch on it's own, then there is no intelligence there. It just "smells" like intelligence.

I tried to argue once that the term "synthetic AI" would be better than AGI, as it removes any notion that artificial implies fake. [Note to surprised self: apparently there is a Wikipedia page describing this exact argument https://en.wikipedia.org/wiki/Synthetic_intelligence]

"Scotsman AI"

This seems like semantic quibbling. All those terms seem synonymous to me.

I feel that there's a legitimate reasons to split them into those terms as there are some significant distinctions that could be made in terms of "level of intelligence" of the AI.

For a much more detailed exploration of this topic, I think Wait but Why's article did a pretty thorough job: http://waitbutwhy.com/2015/01/artificial-intelligence-revolu...

I prefer computational intelligence to AI, so maybe CI?

Strong AI means a different thing.

"Useful AI"?