If you mean hype in media and general public, I agree with you. Research is inherently risky and uncertain, often that is not conveyed correctly. Also research results are often oversold.
If you mean that big tech companies are overinvesting in Machine Learning then I have to disagree. It's not a coincidence that the companies that invested the most in Machine Learning (Google and Facebook) are companies that have a lot of data and already use Machine Learning in their products. It provides things like better feed ranking and better signals for search. These "invisible" improvements to existing products are often overlooked.
And the criticism you note isn't one-directional in the field at large.
Yes, like I mentioned I think there is confusion about terminology. Many ML Engineers (in my definition) call themselves Data Scientists. This leads to misunderstandings, when people don't recognize that others have different goals.
But what is driving this hype, as well as the Blockchain hype, is not engineer-driven companies like Google, but rather MBA-driven buzzword-friendly tech companies (imagine Balmer-era Microsoft), non-tech companies who want to share the cake, the media and finally the investors who are misled by the rest, but end up creating a capital-based feedback loop for them.
As far as the people driving the hype train right now are concerned, Machine Learning is the way you do AI, and Deep Learning is just a more powerful (ahem deeper) version of Machine Learning. This is what AlphaGo used to defeat Lee Sedol, so it's obviously superior, and we should use it to process all data, in the same way we should strive to store everything on a blockchain, which is clearly superior to hash tables and databases.
I'm imagining someone tweeting at Robin, several years back:
"Good Prediction Markets expert says: Most firms that think they want advanced Prediction Markets really just need linear regression on cleaned-up data."
;)
See my comments here:
https://news.ycombinator.com/item?id=13079598
under our recent article " Artificial Intelligence Generates Christmas Song".
Basically, if there is no pixie dust that makes humans intelligent, and instead it is a matter of the architecture of the brain and the first few years of supervised sensory input, then neural net breakthroughs (which use a similar architecture/topology) have the potential at any moment in time to break through and match general human intelligence.
What I mean is that if someone sent back source code from 80 years from now, but we had to run it on a bunch of Amazon / Google servers in a server farm, we're pretty much guaranteed to have enough computing power to do so!
(This is a combination of the number of neurons, their number of connections, and their very slow speed.)
We have the hardware.
Now: we do not actually have the source code from 80 years from now that we can go ahead and run on those machines.
So, we're at like heavier-than-air travel right before the Wright Brothers flew at Kittie Hawk. Except we have like jet engines already -- just no way to design them into something that flies.
I think that AI is vastly underrated and I watch with incredible interest every single breakthrough.
For the second category above, ML built into products or engineering solutions, Alphago surprised me, because Go had an intractibly large possibility space, it's not in any sense subject to brute-forcing or exhaustive search.
Dragon's Natural dictation surprised me in that using its model its able to get basically perfect dictation. I've never worked as a transcriptionist, but a small search reveals it has basically annihilated the industry of medical transcription.
These are not the big, general breakthrough.
But the big, general breakthrough is right there, somewhere. The results researchers are coming up with are astounding, and they're doing it in many cases with neural nets, quite similar to the wiring of the human mind.
The hype is faaaaaaaaaaaaaaar less than warranted for the stage that we're at. At any moment someone can put together something that achieves higher-level intelligence and can be set loose upon the world corpus of culture.
True, there are no clear indications that this is about to happen (For example: people do not extract innate language algorithms from the human genome which encodes them), so we are not exactly taking many steps that we could be trying to. As far as I know we're not even genetically engineering people to see what different parts of DNA do -- which is obviously a very, very good thing, and who would allow anyone to bear to term a child made as an experiment to see what DNA does.
But despite not going from a human starting point, the results that we are achieving in many cases match and surpass human ability - while we do know that in many cases some of their architecture is similar. I feel quite strongly that we have more than enough hardware for general intelligence - and I see advances every day that could end up going past the ppint of no return on it.
--
EDIT: got a downvote, but I would prefer a discussion if you think I'm wrong.
And you're trying to rebut this by referencing an AI-generated Christmas jingle. I think the author rests their case...
Fast forward to the 1700s and artisan engineers in France were creating similar devices that automatically played music. Toys for the entertainment of the aristocracy. Some insightful engineer noticed the use of pinned cylinders to recreate music, and realized the device might also be used to create textiles with a loom. You could program a machine to create textile patterns. While the initial implementations didn't work very well, eventually Joseph-Marie Jacquard created a version that revolutionized textile manufacturing in the early 1800s, relying on punch cards instead of expensive-to-create cylinders.
By the mid-1800s Charles Babbage became interested in Jacquard's work using punch cards to create complex textile patterns. "The major innovation was that the Analytical Engine was to be programmed using punched cards: the Engine was intended to use loops of Jacquard's punched cards to control a mechanical calculator, which could use as input the results of preceding computations." - https://en.wikipedia.org/wiki/Charles_Babbage#Analytical_Eng...
Toys are amazing for inspiration and innovation.
Remember, 10 years ago smartphones were in the realm of science fiction. Now, everyone has one in their pockets.
Today we can train a computer for a week to generate a cute little christmas jingle. How many years do we need to train a human child to do the same? AGI won't happen overnight, sure, but its effects of that research will be felt continuously along the way.
If you used to have 10 people working at a supermarket and you now have 6 with self-checkouts, that's your 40% jobs lost (for that supermarket) right there.
Tesla are describing their 'hardware 2' self driving platform, fitted to all of their new cars, as "full self-driving hardware", and they've posted several videos recently of fully autonomous journeys on public roads. They've said they plan to roll out an OTA update to enable full self driving next year.
Mercedes-Benz have a limited self-driving mode on their 2017 model.
Uber-owned Otto, a company focusing on self-driving technology for trucks, plans to start offering its services next year.
And of course Google is quietly continuing their self-driving project, although they've gone a bit quiet recently.
Now, it does look to me like we've still got a fair few corner cases to straighten out, but either all of these companies are badly wrong, or we'll be seeing self-driving vehicles on sale to the public in the next few years (even if not next year).
IOW I meant to transclude that discussion here. (Perhaps within that comment thread a good specific summary comment is: https://news.ycombinator.com/item?id=13090869)
Obviously it is hard to know when that magic moment will happen that some kind of general AI is created that can learn in some sense similarly to how humans do. My every indication and astonishment at the results that are being produced strongly suggests "at any moment". The results are absolutely astonishing every day and we have vastly more than enough firepower.
You might also be interested in this separate thread where I dealt with questions of consciousness and pain. I again reference it here:
https://news.ycombinator.com/item?id=13026020
and you can click through at the top to follow my reference.
We are way past the point of no return here, and in my estimation it is a question of years or at the most decades - not centuries.
> Obviously it is hard to know when that magic moment will happen that some kind of general AI
> is created that can learn in some sense similarly to how humans do. My every indication and
> astonishment at the results that are being produced strongly suggests "at any moment".
As an IT guy with a basic but solid neuroscience education (which isn't even needed for what I'm about to say): Yep, you are on the hype train, and very deeply. I really like reasonable discussions, I have no idea what this is right here. We will see more amazing results, sure - but applications will be specialized narrow subjects. From creating "some kind of general AI is created that can learn in some sense similarly to how humans do" we are still very far away. Your statements remind me of 1960s "future" hype,a nuclear reactor in every car by the year 2000, stuff like that.>As an IT guy with a basic but solid neuroscience education
-- could you go ahead and take a few minutes (maybe will take you 5-10) to read through my above-referenced links referencing my previous discussion and tell me whether I'm correct in your estimation on the bottom-up aspect - i.e. the amount of computation that human neural nets can likely be doing, and how it compares to server farms with fast interconnects today?
I'm not an expert in neuroscience so your feedback might be helpful there.
The above sentence is true, but it has no bearing on anything.
The fact that they weren't reverse-engineered (yet) would still have huge bearing on everything.
By 7 billion samples I mean the humans walking around. Your analogy with an RSA crack is fundamentally different beccause biology doesn't do it in 3 pounds of grey goo in seven billion different bodies already.
so you would have to come up with an analogy that uses something we cant use, to say, okay fine it exists and fine, we have the hardware to also do it, but the former doesn't have any bearing on us doing the latter.
Still it wouldn't help us find the P-time algorithm in question. We could say "it seems to exist", but that would not imply "we'll discover it any day now".
> If brains routinely cracked RSA
For brains numbers have a completely different meaning and internal representation than for computers. Brains don't "think in numbers". Doing the kind of math we invented is a major effort for the brain, it's not what it developed for, and it is very poorly equipped to do explicit numerical calculations (emphasis on "explicit"). So looking at brains to "crack RSA" seems like waiting for a hammer to be useful in driving screws. > i.e. the amount of computation that human neural nets can likely be doing
We don't know nearly enough what they are really doing! We only know a few selected bits and pieces! You are basing your assumptions on nothing, so according to logic any conclusion is possible from a faulty premise. On which you promptly deliver spectacularly.Also, the "computation" a brain does at one moment leaves out the time aspect: Lots of things lead to constant changes. The wiring changes all the time. The "computation" metaphor has little use for describing or understanding this major aspect of "brain". The more I learned about neuroscience the more unhappy I got with the computing metaphor that I had had going in (as a CS graduate, naturally, I think). The brain is so very, very different from my pre-neuroscience-courses notions.
How much neuroscience do you know? If the answer isn't at least an undergrad introductory course (the accompanying book is over a thousand pages), why do you get the idea you can make any predictions?
Read this to read about complexity in biology vs. engineering and what scientists in the field think how well we are dealing with it:
- http://biorxiv.org/content/early/2016/05/26/055624
- http://www.cell.com/cancer-cell/fulltext/S1535-6108(02)00133...
Test yourself: Do you understand what he's talking about? http://inference-review.com/article/the-excitable-mitochondr...
Fortunately you don't have to sign up at university these days just for such knowledge:
Free courses (if you ignore the certificate nonsense):
- https://www.mcb80x.org/ (This is linked to from edX as "The Fundamentals of Neuroscience" Parts 1, 2, 3)
- https://www.coursera.org/courses?languages=en&query=neurosci... (Especially "Medical Neuroscience": https://www.coursera.org/learn/medical-neuroscience)
You're assuming neural nets are the right model.
Like a 19th century person saying, "if there is no pixie dust ... then eventually Newtonian mechanics will explain these unexpected wobbles we see in the planets' orbits."
Any model that gives reasonably similar results to human thinking will be useful. Planes don't have feathers and don't eat bugs, but we say that they "fly" anyway, and they carry far more cargo / passengers (and faster) than birds ever could.
What we need instead of "consciousness" is a set of concepts:
- "perception" - means representing inputs in such a way as to be able to adapt to the environment to achieve goals (I place qualia here)
- "judgement" - the ability to select the best action for the current state
- "reward" - comes from time to time, and is used as a signal in behavior learning
These three concepts : perception, judgement and reward are simpler, clearer and less ambiguous. They have implementations in AI, not human level yet, but getting there. These concepts make the problem concrete instead of bringing up 2000 years of attempts to "get consciousness" based off armchair philosophising.
A more interesting question than "what is consciousness?" would be: what are the reward signals that train the human brain. We do Reinforcement Learning in the brain, but the reward signal is complex, made of multiple channels, which are evolutionarily optimized for our survival. It's the engine of drive that puts us in motion. We need to reverse engineer that in order to replicate human consciousness in silico.
Consciousness is a replicator, a system that is concerned with survival and maintaining balance in the face of perturbations and entropy. What it does is to make us go find food, make babies and protect from danger - essential actions without which the human species would disappear, and consciousness with it. Or, in other words, it maximizes its rewards over time, reward being having food, shelter, company, access to learning and a few more. But it's just limited number of reward types, and they are much simpler to understand than consciousness itself.
So consciousness is just that thing that maintains itself instead of becoming disorganized and dissipating out like most physical processes that lack a self stabilizing, self replicating dynamics.
Consciousness is a balancing system for the colony of cells that just want to pass their genes into the next generation.
is there anything in that which requires that it be a physical biological substrate rather than a model? Can't computer models follow the same behavior as long as that's how their environment / system is set up? (More specifically, can't they also have brains made up of cells connected to each other, in a way closely analogous or even an exact simulation of chemical pathways?)
But as you said, it's the algorithms that are by far the long pole. Current supervised learning is much akin to simple rote learning. This is the promise of reinforcement learning - and the ability to truly learn on your own through experience. That's scalable. *that said ... I'm biased being one of Rich's students :(
And the criticism you note isn't one-directional in the field at large. I'm finding that ML/AI researchers deriding ML/Data engineers and "scientists" as not doing "real" ML or AI is becoming a thing, similar to how some computer scientists deride engineering as not doing real computing.