There's this: http://www.nickbostrom.com/papers/survey.pdf

>We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.

This survey is a few years old. Discussion and knowledge about AI risk has increased considerably since then. And AI itself has made remarkable progress in that time as well. It's amazing how much progress there has been in AI in just the last 5 years. Who knows where it will be in 40 years. Look where computer technology was 40 years ago.

Is there an established, rigorous definition of "high level machine intelligence"? Or even a concrete list of sufficient criteria? Every single discussion I've encountered leaves the notion undefined, or just "I'll know it when I see it"

Artificial General Intelligence is roughly defined as a machine that can do all the things a human can do. For example, an AI capable of doing AI research and programming computers, would be AGI. The Turing test (as Turing originally described it, not garbage like chatbot competitions) is the most widely accepted standard of AGI.

Superintelligence goes well beyond that. A machine with cognitive abilities far beyond humans. I don't think such a machine is very unlikely even in the near future. It's unreasonable to believe that humans are the pinnacle of intelligence. We are just the very first intelligent creature to evolve. Our brains are heavily resource constrained by size and energy. And neurons are many orders of magnitude slower than transistors. And also far larger and less compact.

The problem is this conception of superintelligence as something that comes after general artificial intelligence. Superintelligence is irrelevant. We'll have superaptitude before general AI, and that will be enough. One of the important principles of design is that specialized solutions are better than general ones.

The tasks humans are hired to perform by employers, and the processes which can be used to break democracy by exploiting big data, are far more constrained than the scope of cognition required to pass the Turing test.

You forget that democracy has always been broken, by design.

>>The conscious and intelligent manipulation of the organized habits and opinions of the masses is an important element in democratic society. Those who manipulate this unseen mechanism of society constitute an invisible government which is the true ruling power of our country. ...We are governed, our minds are molded, our tastes formed, our ideas suggested, largely by men we have never heard of. This is a logical result of the way in which our democratic society is organized. Vast numbers of human beings must cooperate in this manner if they are to live together as a smoothly functioning society. ...In almost every act of our daily lives, whether in the sphere of politics or business, in our social conduct or our ethical thinking, we are dominated by the relatively small number of persons...who understand the mental processes and social patterns of the masses. It is they who pull the wires which control the public mind.<<

The current fit on part of the intellectuals and media is due to them having lost this power to control the narrative and ideas because they failed to adapt to the new media which have replaced the old.

citation for the quote?

Many, maybe even most, interesting tasks in AI can't be done without general intelligence. Specialized AI can only do very limited, simple tasks, that require little thinking or understanding. That's cool, but it's hardly an existential threat like superintelligence is.

This is what I'm talking about. This comment is very vague.

Vague in what way? The Turing Test is a pretty concrete definition of human-level AI, which is what you asked for.

So how will this intelligent thing come about.

Once we have AIs as smart as humans, they can do AI research as good or better than human researchers. And they can make AIs that are even better, which in turn can make even better AIs, and so on.

Dumb evolution was able to create human-level intelligence with just random mutations and natural selection. Surely human engineers can do better. But in the worst case, we could reverse engineer the human brain.

> Once we have AIs as smart as humans

Whether a true 'generalist' AI is possible in the forseeable future is debatable.

> they can do AI research as good or better than human researchers.

Now your AI is not just a 'generalist' but rather a specialist in AI. A very big leap of faith has occurred here. This also presumes that the AIs are even capable of effective invention and improvisation rather than mimicry and optimization (the only two features we have seen from the very best of cutting-edge ML work so far.)

> And they can make AIs that are even better, which in turn can make even better AIs, and so on.

All of which is predicated on the limiting factor being software and not hardware. If it is the latter then these postulated AIs are hitting the same brick wall as humans and thinking about the problem faster or harder does not magically make the necessary hardware appear.

>Whether a true 'generalist' AI is possible in the forseeable future is debatable.

Sure, but see the survey I posted above. The rate of progress of AI is incredible. We will almost certainly be approaching human level in a few decades at most.

>Now your AI is not just a 'generalist' but rather a specialist in AI.

That's what general intelligence is. The ability to learn different specializations. AI researchers are not literally born as AI researchers and capable of nothing else.

> This also presumes that the AIs are even capable of effective invention and improvisation rather than mimicry and optimization

Why wouldn't they be? If they are generally intelligent and can do all the same tasks humans can do. Whats magical about invention that would prevent computers from ever doing it?

>All of which is predicated on the limiting factor being software and not hardware.

All the same arguments apply to hardware. Hardware has been improving exponentially for a much longer time than AI has. And I think hardware may already be close enough. Transistors are orders of magnitude faster and denser than biological synapses.

I am familiar with the survey you presented and can only point out that if you had passed out the same survey back in the late 80s when I was in the field it would have had a similar result. The long-term estimates of people whose current paycheck depends on these long-term estimates being achievable are basically useless BS.

The rate of progress in AI is actually not "incredible" and is in-line with the advances in hardware which have made old research suddenly applicable to a wider range of problems. As someone on the outside looking in it may appear as though magic is happening, but the field is mostly progressing at an only marginally faster rate than it has done over the previous few decades. What has changed significantly to lead to all of these "incredible" results you see is the larger data sets available and improved hardware upon which to run massively parallel but weakly connected computations.

As for why AI can't invent or improvise I am simply suggesting that so far we have only seen optimization and in fact invention and similar feats may actually require more work than we realize. A statistical simulation (based upon a huge corpus) of how a human would respond to various situations is NOT general intelligence, and so far you are just making hand-waving assumptions that paper over a large number of hard problems that no one has a clue how to solve.

As far as I know, there hasn't really been a quantum leap in AI in decades. Most of the fundamentals of the things you see in AI and machine learning are old, like 70s and 80s. The big change is that we have way more and cheaper processing power, so you can see AI happening in fields and areas that were just impossible in the past.

A true Turing Test passing machine, is still something that we seem to be more than a quantum leap behind of.

Who says we need a quantum leap? Most technologies progress by slow iterative improvement. The idea of sudden significant breakthroughs is mostly a myth. Even in biology, the human brain only has some slight differences from other primates. Which in turn aren't terribly different from other animals.

In any case, I think there has been tons of progress since the 80s. Taking the best algorithms from the 80s and running them on modern hardware would fail. The core idea of backpropagation and gradient descent was there, but not much else.

I suspect the "quantum leap" itself will be yielded by a new method of parameter updates, perhaps one that expands the scope of neural networks as they are presently defined.

Maybe that's not the right question. What I understand from the article is that the threat is about being manipulated by computer algorithms. We don't really care the computers are actually intelligent and thinking or whatnot. What's bad is that we are manipulated by computers and they automates more and more areas of our lives each year.

How about: "It'll know you when it sees you"

futurism as a whole is 100% un-scientific

And does being unscientific make a thing wrong? Saying "we won't have strong AI in our lifetimes" is an equally strong claim that is also unscientific. You can't say "we can't know" because what does that even mean? That the outcome is 50/50? That's still pretty decent odds for something so significant!

As a professional scientist in AI and deep learning, I see futurists making wild predictions about AI that I wouldn't see anyone serious in the field coming remotely close to making. I don't know how it is with other futurism fields like genetics, space travel and energy generation, but I expect that it's a similar situation.

All I'm trying to say is, let's please keep a cool head and a wholly skeptical approach about all this.

But the survey I posted that started this discussion was a survey of experts in AI.

Note that no one is claiming that skynet will appear any day now. Just that we will very likely have human-level AI in a few decades. I don't think that's a terribly wild or speculative claim.

also please note I didn't say "we won't have strong AI in our lifetimes". Let's work on the actual developments specific of science instead of what we dream it to be. We want people to think about deep learning as a sophisticated pattern analysis algorithm instead of thinking that Skynet is going to start a coup any day now.

I wouldn't place too much stake in predictions made by experts. There is evidence that these may be far less accurate than we'd like to think, even for very near events. See http://press.princeton.edu/titles/7959.html. This research was also discussed in Thinking Fast and Slow.

Individual experts may make very inaccurate predictions, sure. But averaged together you get the wisdom of crowds effect.

Only if those experts' individual inaccuracies are uncorrelated - which for public intellectuals, they will probably always be. See https://en.wikipedia.org/wiki/The_Wisdom_of_Crowds#Failures_...

>Who knows where it will be in 40 years. Look where computer technology was 40 years ago.

The Commodore PET, Apple II, and TRS-80 all came out exactly 40 years ago (1977)[1]

[1]https://en.wikipedia.org/wiki/1977_in_science#Computer_scien...

In the risks we should take into account problems caused by mass unemployment (caused by AI).

By the way, there is a list of who does and who does not think there is a serious risk for society (of any type) within the next 50 years: https://goo.gl/Oe7S6e

Consider that 41% of those surveyed conclude that machines will never be able to "simulate learning and every other aspect of human intelligence". The next 41% conclude it is possible within an unbounded "more than 50 years". It seems to me that the vast majority of experts are pretty skeptical.

That's not 41%, that's 4.1%. When asked when there would be a 90% of AI, the median answer was 2075.

No. It's definitely 41%. Look at page 6 of your link

    The earliest that machines will be able to simulate learning and every other
    aspect of human intelligence:
    
    Within 10 years 6 5%
    Between 11 and 25 years 3 2%
    Between 26 and 50 years 14 11%
    More than 50 years 50 41%
    Never 50 41%
    Totals 123 100%

That's in the prior work section. It's a much older survey of one specific conference.

And was the other survey a comprehensive survey of everybody?

The divergence in poll results seems to suggest we should put even less credence on polls here than usual.

It was in 2006 which is way before AI started seeing it's recent progress. And it was a conference full of old fashioned AI people.

I think you've missed the point of the article. It's not completely about AI per se, but about the broader issue of technology in general bringing us to new territory with respect to the ability to control and/or manipulate society, which could have disastrous unintended consequences.

Therefore, we need to be proactive about policies that would protect against unfettered influence by those controlling the technology.

You've said the equivalent of "guns don't kill people. People kill people".

There are numerous historical examples of societal manipulation (of the most extreme kind) that had little to do with technology.

I don't disagree about the need for policies; I just think those policies need to be directed toward humans as the weak link in the chain, not machines. This is not like gun control where a single person with a weapon can do a lot of unchecked damage, and thus access to guns needs to be controlled. Very few have access/can do damage with an algorithm.

Which is the meat of the article.

This is definitely the point to make a "did you read the full article" check.

My reading of it was that people were clearly seriously concerned about the current state of the art. The article also considered Future possibilities.

We don't need true artificial intelligence. Just pseudo intelligence will kill off a lot of jobs. The most hurt will be the developed countries. Let just say a few million driving jobs go because of Ai. That will snowball and remove millions of other supporting jobs. Billions of dollars will stop circulating in the economy as most of it will go into the pockets of corporations and rich people who can afford to owns stocks of these companies. Instead of being used.

Or, as it stands today, human decision makers using the output of these algorithms as a signal, without understanding the potential flaws of that signal. "I just did what the computer recommended"

Someone (can't remember who) said it better: "You don't want to be an edge case in this brave new world."

"I was just following orders" the excuse for most of the worst behavior of humans.

The difference is they have the power to make a decision.

"Why did the computer recommend that?"

"No one knows, not even the programmers who wrote it! Deep learning is a black box!"

https://en.wikipedia.org/wiki/The_Machine_That_Won_the_War_(... - Asimov, 1961

> Rather, machine-learning algorithms are trained on a diet of human-derived data that is simply a reflection of existing human biases.

Maybe we shouldn't call Supervised learning techniques "Artificial Intelligence", but instead "Memoized Intelligence"?

Then again, it's probably a spectrum from memoized intelligence to "true" intelligence.

Fuck, I am also trained on a set of human derived data and past experiences. What even is intelligence?

Totally agreed; intelligence (IMHO) is all about using pattern matching to model and reason about a domain. Social things like us get some of that acquired pattern matching knowledge from other social things.

I guess the difference is that supervised learning systems can't yet make an autonomous effort to acquire new knowledge from other social things.

I agree with you on the first points. We don't know how soon computers will match us in intelligence and they're still underpowered but 2 things might accelerate their rise: 1. GPU computing. Much more similar to the brain. 2. IoT - it might bring computers the senses humans have today. A self-correcting ML system with access to broad sensors, huge computing resources and the Internet is starting to look surprisingly similar to our abilities and data access.

Without disagreeing (nor agreeing), what do you mean by intelligence? A lot of the disagreements I see is from people disbelieving that "mere" math and algorithms can exhibit intelligence. A mystical sort of Intelligence of the Gaps. I don't think that is your stance but I'd be interested in clarification.

the thing is that deep learning is like 3 years old. I'm 28 years old and even I have seen the world without the internet, forget about cellphones and smartphones

What I mean to say is, progress is incredibly, impossibly blazingly fast. 2060 is extreeeemely far away.

Deep learning is at least 50 years old, depending on when you want to start counting. Take a look at Ivakhnenko and Lapa's paper "Cybernetic predicting devices" from 1965, that had the first algo. There's been a lot of work since then during the 70s, 80s, 90s, through today.

In fact the idea goes back even further to the 1940s, when the first papers were published about how NNs could possibly work.

Edit: found this page that has the history: http://www.scholarpedia.org/article/Deep_Learning

>Rather, machine-learning algorithms are trained on a diet of human-derived data that is simply a reflection of existing human biases

The AI that plays Go is now better than all Go Players and it trains by playing itself. It was bootstrapped by analyzing human games, but now it's past that. Theoretically reinforcement learning AIs could have access to deeper levels of understanding than just by copying humans because they can build upon experience. These are limited domains for now, but I think the first startling AI will come from chatbots that can come up with novel arguments. Watch the chatbot space. When someone comes up with a really good chatbot that can come up with creative responses, that will really be an AI turning point. All the deep learning ones I've seen so far are like talking to someone with moderate dementia. They forget things from earlier in the conversation and go into conversational loops.

Go is an objectively defined game; anything dealing with the real world is going to involve subjective human biases in the criteria.

AI beat humans at poker, which is a game with incomplete information.

https://www.theguardian.com/technology/2017/jan/30/libratus-...

"The algorithms that power Libratus aren’t specific to poker, which means the system could have a variety of applications outside of recreational games, from negotiating business deals to setting military or cybersecurity strategy and planning medical treatment – anywhere where humans are required to do strategic reasoning with imperfect information.

“Poker is the least of our concerns here,” said Roman V Yampolskiy, a professor of computer science at the University of Louisville. “You have a machine that can kick your ass in business and military applications. I’m worried about how humanity as a whole will deal with that.”

Poker may have incomplete information, but it's still an objectively-defined game. The cards, the "win" state, all of that stuff is trivially encodable and well-defined.

"...machine-learning algorithms are trained on a diet of human-derived data that is simply a reflection of existing human biases."

This one point simply cannot be stressed enough.

Glad your reply is top. To be honest we could do with less of these fluff pieces on HN, leave them to Reddit and Facebook.