I agree but I am more pessimistic about it. There are 2 inter-related factors that will stop AGI from occurring at the level we all dream of: Competition and Specialization. An AGI is not something that will occur in a vacuum. It will be developed and created in the world we all are living in. The one with dirt and mortgages and greedy people. More importantly, it will come about in a world where half of the people are dumber than average. The AGI does not need to solve the Riemann-Zeta hypothesis. It just needs to afford to have it's owners/creators bills paid and maybe make some on the side for profit. That is all that it needs to do. Yes, it can do more, a lot more, but it only needs to be a little bit smarter than the average human so that it's owners can scam us by proxy. Look at the Ashley Madison hack. You had real human men paying real world dollars to suss out if they were talking to a bot or not. That's all that AGI really needs to do to sustain itself over a short time interval. How long a time interval do you want it to go for will then add the complexity and get it to 'human' levels of general intelligence as it tries to deal with rarer and less common failure modes (like itself getting hacked in the Ashley Madison case). AGIs will have to fork out of the super-targeted AIs that we now have that make monetary sense. That will take a lot of time and money to do this, and the returns on the investments are uncertain to me.

> People who ask questions such as this often don't consider that it remains eminently possible that AGI is an impossibility for us to build.

I 100% agree that AGI is probably a long ways off barring some surprise (not that such a surprise is impossible, but definitely not expected). That said, the fact that the human brain works means that brains can be made. Unless you think there's something literally magical going on in the brain, then the brain is just a physical thing, and other physical things with similar behavior can be constructed. Maybe we won't be able to do it with current computer hardware, but some day we'll manage it.

I've never heard an argument that AGI was impossible that didn't basically boil down to "No, we're special, and you can't prove otherwise." Not a very convincing argument.

> "No, we're special, and you can't prove otherwise."

More the opposite. What makes you assume we're smart enough to fully understand how it is that we work and reproduce it non-biologically? My main issue is around some people treating it as an inevitability as opposed to a possibility. I mean I'm all for humans performing crazy engineering feats but we have to ask ourselves if some feats are beyond us. Could we build a star? Would we build a star? Will we build one 2017?

You get me?

> What makes you assume we're smart enough to fully understand how it is that we work and reproduce it non-biologically

Absolutely nothing. I believe we'll eventually make artificial intelligence, and I doubt we'll really understand it when we make it. Human intelligence evolved naturally as a result of a statistical process. When we make artificial intelligence it's totally likely that it will be the result of some human controlled statistical process that looks nothing like traditional software.

> Could we build a star?

Almost certainly eventually.

> Would we build a star?

Probably not. But building a star would require vast resources, and I'm not sure what we'd get out of it. There are lots of stars. Artificial intelligence really isn't comparable on this front.

Is it not possible that we could simulate a star though? Not in 2017, but at some point?

I'm of the opinion that no, we will not. The computing power required to simulate a star (or a human brain, to go back down the analogy chain) is not there and will not be there, without some quantum computing magic. I doubt the sun provides enough energy for us to fully simulate the sun.

Simply running basic neural network is slow work, even with the best supercomputers we have today. Throw in the fact that a neural network is a super simplistic model of how we currently believe synapses work, and the scale of the simulation problem gets hard, fast.

> I doubt the sun provides enough energy for us to fully simulate the sun.

That's an interesting way to put it! I tend to agree that simulating a human brain will need more than the computational resources to train a neural network with 1 billion neurons (if those resources will ever be available, which is questionable). That's not to say we won't find a way around that problem though. Of course, all the worry about a dangerous AGI hinges on this premise, and how we get there is at best unclear at the moment.

sure but we haven't even started to define what constitutes a valid simulation. That's half of the point here, we're trying to recreate something that we don't know.

That's true. The big if is whether we can ever have the resources to fully simulate the human brain. If we ever get close to that, then we can worry about the "intelligence explosion" and so on. It feels very possible that we'd work out how the human brain works at some point, though.

What is the strongest argument for impossibility being a possibility?

There are arguments, notably in "The Emperors New Mind", that assert that consciousness must be based on quantum mechanics in some ill-defined way and therefore that "algorithmic" computers can't therefore be conscious.

https://en.wikipedia.org/wiki/The_Emperor's_New_Mind

NB I didn't find this argument particularly strong, but it's a long time since I read it. The brain would have to be doing something spectacularly strange for it to be impossible for us to emulate in one way or another.

Out of context it indeed sounds rather fuzzy and weak. It's not even intuitive to me that consciousness comes first. Maybe they are even independent. I should probably read the book though.

More generally, I don't think invoking quantum effects is an especially strong argument for impossibility in the general case. We already use quantum effects to some extent and make progress on quantum computation. It may be evidence of greater difficulty, but not of impossibility. Unless there is a strong case for some particular quantum effect being unharness-able.

"It may be evidence of greater difficulty, but not of impossibility"

Pretty much everyone I've talked to who knows about AI (including a former Penrose student who was a classmate on my postgrad AI degree) strongly refute the claims in The Emperor's New Mind. I believe Penrose was at Oxford when he wrote it (in 1989), in which case he would surely have come across David Deutsch's universal quantum computer (from 1985).

The argument is particularly poor considering that normal Turing machines can simulate quantum TMs (albeit with an exponential slowdown).

At the scale an actual brain works, an actual exponential slowdown because of "quantum brains" would make the problem flat-out impossible. An actual human brain has an average of 100 billion neurons. To simulate an actual brain, you'd get an 'additional' slowdown of 2^100,000,000,000. The problem would go from possible in maybe 20-50 years to impossible unless we manage invent some sort of quantum/neural computer.

Not that quantum mechanisms in brains are likely, so probably not an issue.

That's approaching it from the wrong angle. That's like saying "What's the proof that unaided human flight is not possible?" (Edit: Note the unaided part. As without a plane or something. I guess I should have chosen something less easy to confuse like walking through walls or something.)

The issue is that we do not have proof that general AI is possible. We conjecture that it is based on the assumption that our brains could be simulated by enough artificial neurons, (or some other method) but we don't know that for a fact.

And so far we can't even fully simulate a mouse brain, let alone a human brain. We can make something that imitates aspects of mouse behavior, but we can't actually simulate a mouse brain.

All the hype around General AI being "just over the horizon" right now is just like it was back in the 80s (or the 60s.) It's a bit over the top. If you look at it from a critical perspective there isn't a lot of solid reason to be so bullish.

I'm not following completely – surely there were people arguing that human flight was impossible, without strong reasons, and they were wrong? EDIT: missed "unaided". I think you can make a strong case for that impossibility that isn't being made against GAI.

Based on everything else we know about the world now, we can "just" simulate a brain. "Just" may mean "after another 200 years of work", of course. The alternative is "the brain is magical somehow", which is not a strong argument on its own – we don't often end up finding magic to be the answer.

Where do you see hype about General AI being just over the horizon? Most popular discussion I read is about two things: a) the replacement of human work by increasingly-improved specific AI; b) the possibility of rapid advancement of general AI once invented. I haven't seen anyone say b) is imminent – only that it is likely to be fast/dangerous when it happens.

> Based on everything else we know about the world now, we can "just" simulate a brain. "Just" may mean "after another 200 years of work", of course. The alternative is "the brain is magical somehow", which is not a strong argument on its own – we don't often end up finding magic to be the answer.

In order to accurately simulate a particular brain, we'd need the technology to (i) accurately read all the information encoded in a particular brain, which everything we know about the brain suggests depends on getting a snapshot of the state of 85 billion neurons - and chemical interactions of those neurons' constituent proteins - at a particular millisecond in time. (ii) manipulate matter (or write matter-simulating code) such that we duplicate the state of that brain at that time, and probably the associated body too, with sufficient accuracy for the biochemical reactions (or digital emulation of the biochemical reactions) continue to produce a patterns resembles the thought processes of the emulated brain in their outputs.

I don't think we need to invoke "magic" or vitalism or quantum brains to suggest that replicators capable of reproducing a particular state of a mind-bogglingly complex set of biochemical processes without significant information loss may not be a practical possibility in future. If anything, that sounds harder than achieving AGI or something approaching it via other means.

As for the hype about general AI, we're responding to a thread musing on whether we'll get "close" to achieving it this year.

My comment was in the context of GAI feasibility - possibly "the brain" would have been clearer than "a brain"? I didn't want to exclude non-human/non-mammalian brains, even if very similar. Simulating an existing, specific brain was not my intention: that seems like it would involve - as you say - a rather different and larger set of problems.

If "all the hype" is a single HN question (not even an assertion) this counts as "over the top", then what would be an appropriate amount of discussion other than none at all?

Simulating "the brain" for any non trivial brain[1] sounds even harder than simulating "a brain" (or creating AI in some non-brain-resembling form) since it would appear to imply identifying what the billions of constituent parts of a brain do well enough to generalise rather than merely copy patterns of brain function.

Someone who probably isn't wholly uninformed asking if we're getting close to AGI this year is obviously a symptom rather than the cause of the hype. I mean, if we're working on Ray Kurzweil's Singularity is Near timescale, I'll be able to buy computational power equivalent to the human brain for $999 in three years' time. Still, Peter Diamindis did update that a couple of years ago by pushing that deadline out to 2025, so I guess at least I'll still be smarter than my next bottom-spec Macbook Air.

[1]C elegans is pretty trivial

> If "all the hype" is a single HN question...

It's not, though. It comes up over and over here, and often it's a link to an external article rather than an "Ask HN".

I can't recall a single instance of a story or serious comment. Do you have some example stories?

https://hn.algolia.com/?query=artificial%20general%20intelli... shows a dozen articles in the last year, so an average of one a month. I only searched on "general artificial intelligence"; other search strings might find additional articles.

Most have no comments/less than 5 points or only happen to contain the word "general". Many are simply asking questions such as "what does GAI mean?" and "is it possible at all?". You can count that as hype if you want, I guess.

I also wish all of the sugar beverage hype would stop (https://hn.algolia.com/?query=coca%20cola&sort=byPopularity&...).

> Based on everything else we know about the world now, we can "just" simulate a brain.

Right, we think we can. But we don't know. I would argue that as we get closer to doing it, it will become a fact that we can (or we will find that we can't.) The fact that we still don't know for sure that we can (or cannot) is an indicator or a symptom of how far away we are from actually doing it.

Are there examples of similar things where we have failed? i.e. nature gives us an example, we expend massive resources on it over a long period of time and cannot even make substantial progress? Probably I'd just say "tech in that area is only 10 years old" ;)

Is the argument that we're still just hitting the low-hanging fruit of the technological age, so have false confidence in all the things we've equalled or beaten?

I guess what I'm trying to get at is that most of the hardest things may not be logically possible (c.f. incompleteness theorem) or may not be physically possible (c.f. time travel). Having such clear physical possibility (and at multiple levels of complexity/advancement that only arose through selection and mutation) seems like strong evidence we will be able to do it eventually. A claim of impossibility is not illogical, but requires a stronger argument than "there's always a small chance".

Flight is where one argument for the possibility of general AI comes from: the assumption is that we can do whatever nature has demonstrated to us. This argument makes the process seem trivial, though. Rephrasing it to use fusion as the example instead of flight is more realistic. Even so, we know how fusion works and have no idea about intelligence.

It's a very long way off.

I think it's entirely possible it's too complex a problem for us to solve. After all, if you think about it by definition a more intelligent being than us would be able to solve problems we can't. Maybe this is one of those, so there might not be any way for us to bootstrap to a general AI.

I don't really buy that argument (even though it's mine, I've not seen anyone else argue this). We design systems so complex and built from learning algorithms and neural nets such that ultimately we don't actually know precisely how they are doing what they do. We could brute force it by evolving a solution maybe. But it's at least plausible that we won't be able to engineer and design an AI architecturally because we're not up to the task.

my point being that some people treat AGI as an inevitability and I think we're a fair way off a proof.

It's that possibility is not an impossibility.

Thanks for your answer, I think on the same line as you, but I was here with a colleague that believes it's soon possible so we wanted to check what the community feels like it's gonna happen

Sorry to be harsh, I just tend to be harsh as a push-back to the seemingly exceptionally popular belief that AGI is inevitable.

Its just lots of that stuff from the waitbutwhy article really rankles me: e.g. having an x-axis for "human advancement" and then only valuing recent developments while under valuing things like writing or the printing press or treating intelligence as an integer value where an AGI can keep multiplying its intelligence and become completely unassailable by humans. There's a lot of bollocks intertwined with people that hope for AGI and it upsets me. I feel like it devalues and undermines the current progress that is being made.

Assuming it's this one: http://waitbutwhy.com/2015/01/artificial-intelligence-revolu... I don't think the waitbutwhy article rankles so much as makes me scratch my head and think "is this intentional parody?"

I'm particularly fond of the Back to the Future example of how rapidly and unpredictably technological progress was accelerating, which completely ignored the fact that Back to the Future had a sequel whose tongue-in-cheek version of 2015 based entirely on 1980s culture is pretty much dead on, bar the hovering/flying stuff we haven't got around to inventing yet.

What rankles me about waitbutwhy is that graph. It claims it's trying to show how exponential growth takes off faster than you expect based on the past (which is in fact a valid point). But it ruins it with that kink in the graph, which is completely untrue to exponential growth. It conveys the impression that "things are about to really take off just around the corner, but it's false to the actual argument being made.

In other words, he's lying with a graph in order to try to make his argument have more emotional impact. And I have a policy of taking lying as a serious disqualification of people who are trying to persuade me...

what rankles me even more is that they're pretending to make a science based article with facts and logic founded upon a graph that is neither. Its offensive.

I've been reading breathless articles explaining that AGI is going to happen "real soon now" since the early 1980s - to a certain extent I actually believed this and this is what motivated me to get into AI research in the late 80s and early 1990s, which was pretty close to the point where "good old fashioned AI" was dying out.

My own view is that it's highly likely that we'll have something like AGI - but we might also find that it's not actually that useful, or really necessary, and anyway I suspect it will take a long time to get there (I'd guess 100-200 years).

I think you have the right to be harsh when you see all these posts by people who don't have a strong knowledge on AI and you strongly believe in what you're saying and you have expertise in it. As I said I'm on your way of thinking! Thanks again for your answer