One of the many good points that Lanier made was that there is not currently an inevitable, moore's-law-like progression that will take modern AI methods towards the "superintelligence" that is meant when AI is discussed as an existential threat.

Coupling that with the fact that any AI will be limited by its human-defined interface leaves me pretty unconcerned about AI, even if we ever figure out what truly makes a conscious, intelligent being and determine how to replicate it fully in some other medium.

That said, I appreciate Bostrom's work as a philosophical rather than practical matter.

Taking a best-fit line like Moore's law as a fundamental law of the universe is a classic mistake.

Of course, there's no such thing as an inevitable progression of progress in AI research. Until we have AI software capable of contributing to AI research, of course.

"Certainly it isn't human level now, and it might take a decade or two to get there."

Any chance you could define "human level" in a way to make this a meaningful sentence? Twenty-five years after I was introduced to this problem and I still have never heard a better definition of "intelligence" than the Turing test, and it's a purely operational, "I'll know it when I see it" thing.

I.J. Good: "Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an "intelligence explosion," and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control."

A merely human-level AI is one that can match to some window of performance a random human in the intellectual outputs of that human, which are vast and general -- playing games, writing poetry, critiquing film, building small rockets, having conversations, programming, carving wood, teaching a class, diagnosing and fixing a leak, constructing mathematical models... For things that require a body, it suffices to instruct a human what to do, but this thing might as well help design a robot body that's at least as versatile as a human body since we humans do that too even if Atlas is still pretty far off. We're unlikely to have an AI perfectly human-equivalent in the sense that it would probably by default already be superior in some ways by virtue of its ability to not need sleep, do calculations efficiently and correctly, have access to perfect memory, and so on, and unless it's ultraintelligent then it's probably not better than a random human at everything (say directing a movie) just like you aren't.

> it would probably by default already be superior in some ways by virtue of its ability to not need sleep, do calculations efficiently and correctly, have access to perfect memory, and so on

Or maybe we'll learn that in order to be intelligent the machine needs to sleep, do mistakes, forget some things, etc.

Of course, hence probably. That would be really weird and unexpected though.