The animal intelligence researchers face a similar problem.
Turing's test is useless for real AGI, because human-level intelligence is embodied. That means a human-equivalent AGI has to be able to improvise solutions with physical objects, parse body language, generate body language, understand social and cultural expectations in different situations, and do all of this while recognising and generating everyday speech in at least one human language.
These are all defining basic skills for humans. Literally every school age and over human can do them at a basic level. And gifted or exceptional humans can take these "simple" challenges to very advanced levels.
A chatbot doesn't come close to approaching them. Nor does a chess-bot or a go-bot. Nor does an equation solver.
AGI will fail until AI research stops trying to build a better AI research nerd, and starts trying to understand what applied human intelligence looks like in the wild.
This is the other side of the same coin. No. These "defining basic skills" aren't defining anything except if you're going to be a fascist who eradicates people that defy their beliefs about embodiment.
Understanding social expectations is tricky and lots of humans can't do this at all. They require lifetime care as a result, but they are still definitely human and there's no reason to believe they lack general intelligence although it may be stunted if their problems make it hard to undertake activities that let intelligence thrive.
Although human infants do try to bootstrap language, the bootstrap process will not spontaneously produce a working human language from zero in the absence of exposure to existing human language. Humans brought up without language (typically through extreme circumstances because doing this experimentally would be unethical) create a proto-language but not a full-blown human language. We _think_ that a few generations of humans otherwise unexposed to external culture would turn this into a full-blown human language through some mechanism akin to creolisation but we can't check because - as mentioned - it would be grossly unethical. So, there definitely are humans (though not many) who don't recognise and generate "everyday speech" even taking that very broadly to include sign and the procedure used to talk to deaf-blind people.
Elderly people also often cease to be able to generate speech, or give no sign they continue to understand it, while continuing to apparently be intelligent.
I fully agree with your comment, just wanted to point out that imho the quoted paragraph also says a lot about how most of today's researchers totally disregard "human studies" (the "human intelligence in the wild" that you mention), as long as it's not quantifiable/turned into mathematics it has almost no value for them. Which, of course, it's a total bastardization of what even science-oriented people like Popper were writing about.
Not sure that's what you meant but for me yes, the moment when a "machine" would be able to make genuine jokes or when it will become self-aware of its "humiliating" us then I'll agree that we have reached true AGI.
To reiterate, I cannot agree how important it is for AI system to posses "the ability to make jokes/to understand double-entendres", right now that's what makes companies like FB or Google perfect targets for the media and public opinion (that and their total disregard of privacy issues, of course). A true AGI would have to understand the meaning of a green frog or of a cartoony bear by context alone.
And it has nothing to do with human pride or not facing the facts.
I mean by that same logic you could argue we solved AI when we invented the calculator. Because it's far better at calculating than us.
But the moving goal posts are a symptom of the problem that inspired Turing here. The humans will claim they're still the only ones with real intelligence even after they fail that test, but they'll know what they're doing. As we see in this thread, today plenty of people are convinced that they aren't moving the goal posts, even though they are. My suspicion is that the humiliation of the "imitation game" stops that, even as the outward pantomime continues. Humans will keep _saying_ they know they're special but they won't really believe it any more. Good.
When a Pope insists they are certain God exists, I know they're lying, whereas when some little old lady in a sleepy rural village insists the same perhaps she really believes. The Pope has seen enough to know he isn't sure, lie to me as he might.
But yes, AGI is the holy grail of AI research. Currently all AI successes have been training computers to be really good at one thing, chess, go, driving etc... And in many of these domains AI has outperformed humans by a large margin.
But take those AI systems and ask them to do something else outside of their specialty and their knowledge doesn't transfer. In essence they are really just optimized functions for a very specific set of inputs.
AGI would be an AI that increases in intelligence across multiple domains and can respond to novel problems (like a human)
note: you could argue the real definition of AI is actually AGI but that is a different discussion :)