News for hackers. Stuff that matters.
Re:This AI Boom Will Also Bust by sidlls on Saturday December 03, 2016 @02:26PM (#13097083)
It's also suffering from hype.

And the criticism you note isn't one-directional in the field at large. I'm finding that ML/AI researchers deriding ML/Data engineers and "scientists" as not doing "real" ML or AI is becoming a thing, similar to how some computer scientists deride engineering as not doing real computing.

Re:This AI Boom Will Also Bust by c3534l on Saturday December 03, 2016 @07:45PM (#13098233)
This and the above comment kind of says my thoughts on the matter. I think there is a very real emerging field that combines mathematics, traditional computer science, and AI into a big family resemblance of quantitative methods and research, but that we're still in the early stages and we're still sorting things out. I remember hearing someone on hacker news joke that when he got out of school people called him a statistician, then people started saying he was a data miner, and now he sells himself as a data scientist, but he hasn't actually done anything differently apart from just keeping up with the field. Sometimes it feels as if we're just playing with words and people get excited for advances and what it means for the future, but the reality is people just keep plodding along make incremental progress, finding new ways to leverage the tools they have, and continuing to publish papers. For all of the tech industry's worship of "disruptive innovation" and massive changes, the reality is that we've been developing and refining our quantitative abilities and figuring out automation since the industrial revolution. AI may become less exciting to people 5 years from now, but we will still continue doing what we've been doing.
Re:This AI Boom Will Also Bust by ma2rten on Saturday December 03, 2016 @04:14PM (#13097526)
It's also suffering from hype.

If you mean hype in media and general public, I agree with you. Research is inherently risky and uncertain, often that is not conveyed correctly. Also research results are often oversold.

If you mean that big tech companies are overinvesting in Machine Learning then I have to disagree. It's not a coincidence that the companies that invested the most in Machine Learning (Google and Facebook) are companies that have a lot of data and already use Machine Learning in their products. It provides things like better feed ranking and better signals for search. These "invisible" improvements to existing products are often overlooked.

And the criticism you note isn't one-directional in the field at large.

Yes, like I mentioned I think there is confusion about terminology. Many ML Engineers (in my definition) call themselves Data Scientists. This leads to misunderstandings, when people don't recognize that others have different goals.

Re:This AI Boom Will Also Bust by unscaled on Saturday December 03, 2016 @11:15PM (#13098758)
You mean, some tech companies understand the difference between Data Science, AI in general, ML and Deep Learning and try to utilize the best tools in hand to handle their data.

But what is driving this hype, as well as the Blockchain hype, is not engineer-driven companies like Google, but rather MBA-driven buzzword-friendly tech companies (imagine Balmer-era Microsoft), non-tech companies who want to share the cake, the media and finally the investors who are misled by the rest, but end up creating a capital-based feedback loop for them.

As far as the people driving the hype train right now are concerned, Machine Learning is the way you do AI, and Deep Learning is just a more powerful (ahem deeper) version of Machine Learning. This is what AlphaGo used to defeat Lee Sedol, so it's obviously superior, and we should use it to process all data, in the same way we should strive to store everything on a blockchain, which is clearly superior to hash tables and databases.

Re:This AI Boom Will Also Bust by shas3 on Saturday December 03, 2016 @05:51PM (#13097882)
This is the first time I'm hearing that computer scientists deride engineering as not-real-computing. Any references? It doesn't check out based on my background. In fact anecdotally, I've heard the reverse. I've heard EE and stat algorithms folks criticize CS ML/CV researchers for using algorithms as black boxes compared to the rigorous standards of EE/Stat (aka reviewer standards in IEEE Transactions in Information Theory, IEEE Trans Sig Proc, majority of stat journals). Now obviously there are a vast number of CS researchers who are competent at stat-level rigor. But there are also a large number of ML researchers (applied ML?) in CS departments who use algorithms as black boxes.
Re:This AI Boom Will Also Bust by swordswinger12 on Saturday December 03, 2016 @08:48PM (#13098383)
The prof of my applied ML class this semester jokingly refers to ML research's approach to statistics as "punk stats", which I find to be pretty accurate.
Re:This AI Boom Will Also Bust by liviu- on Saturday December 03, 2016 @07:24PM (#13098172)
My understanding is that the grandparent comment was referring to _software_ engineers.
Re:This AI Boom Will Also Bust by cma on Saturday December 03, 2016 @04:10PM (#13097504)
>It's also suffering from hype.

I'm imagining someone tweeting at Robin, several years back:

"Good Prediction Markets expert says: Most firms that think they want advanced Prediction Markets really just need linear regression on cleaned-up data."

;)

Re:This AI Boom Will Also Bust by logicallee on Saturday December 03, 2016 @03:11PM (#13097287)
It is not suffering from hype. There is too little hype. People are vastly underestimating what is about to happen.

See my comments here:

https://news.ycombinator.com/item?id=13079598

under our recent article " Artificial Intelligence Generates Christmas Song".

Basically, if there is no pixie dust that makes humans intelligent, and instead it is a matter of the architecture of the brain and the first few years of supervised sensory input, then neural net breakthroughs (which use a similar architecture/topology) have the potential at any moment in time to break through and match general human intelligence.

What I mean is that if someone sent back source code from 80 years from now, but we had to run it on a bunch of Amazon / Google servers in a server farm, we're pretty much guaranteed to have enough computing power to do so!

(This is a combination of the number of neurons, their number of connections, and their very slow speed.)

We have the hardware.

Now: we do not actually have the source code from 80 years from now that we can go ahead and run on those machines.

So, we're at like heavier-than-air travel right before the Wright Brothers flew at Kittie Hawk. Except we have like jet engines already -- just no way to design them into something that flies.

I think that AI is vastly underrated and I watch with incredible interest every single breakthrough.

For the second category above, ML built into products or engineering solutions, Alphago surprised me, because Go had an intractibly large possibility space, it's not in any sense subject to brute-forcing or exhaustive search.

Dragon's Natural dictation surprised me in that using its model its able to get basically perfect dictation. I've never worked as a transcriptionist, but a small search reveals it has basically annihilated the industry of medical transcription.

These are not the big, general breakthrough.

But the big, general breakthrough is right there, somewhere. The results researchers are coming up with are astounding, and they're doing it in many cases with neural nets, quite similar to the wiring of the human mind.

The hype is faaaaaaaaaaaaaaar less than warranted for the stage that we're at. At any moment someone can put together something that achieves higher-level intelligence and can be set loose upon the world corpus of culture.

True, there are no clear indications that this is about to happen (For example: people do not extract innate language algorithms from the human genome which encodes them), so we are not exactly taking many steps that we could be trying to. As far as I know we're not even genetically engineering people to see what different parts of DNA do -- which is obviously a very, very good thing, and who would allow anyone to bear to term a child made as an experiment to see what DNA does.

But despite not going from a human starting point, the results that we are achieving in many cases match and surpass human ability - while we do know that in many cases some of their architecture is similar. I feel quite strongly that we have more than enough hardware for general intelligence - and I see advances every day that could end up going past the ppint of no return on it.

--

EDIT: got a downvote, but I would prefer a discussion if you think I'm wrong.

Re:This AI Boom Will Also Bust by jpatokal on Saturday December 03, 2016 @03:39PM (#13097402)
I didn't downvote you, but the TL;DR of the article is that most ML demonstrations to date have been toys and there are no known real-world applications that would justify the "40% of jobs lost!" hype.

And you're trying to rebut this by referencing an AI-generated Christmas jingle. I think the author rests their case...

Re:This AI Boom Will Also Bust by ryoshu on Saturday December 03, 2016 @05:21PM (#13097782)
A lot of revolutionary technologies begin as toys. In the 9th century two brothers created a machine called, "the instrument which plays by itself." It uses a rotating cylinder with "teeth" that triggers keys which were encoded to play music. One can say it's a programmable music machine.

Fast forward to the 1700s and artisan engineers in France were creating similar devices that automatically played music. Toys for the entertainment of the aristocracy. Some insightful engineer noticed the use of pinned cylinders to recreate music, and realized the device might also be used to create textiles with a loom. You could program a machine to create textile patterns. While the initial implementations didn't work very well, eventually Joseph-Marie Jacquard created a version that revolutionized textile manufacturing in the early 1800s, relying on punch cards instead of expensive-to-create cylinders.

By the mid-1800s Charles Babbage became interested in Jacquard's work using punch cards to create complex textile patterns. "The major innovation was that the Analytical Engine was to be programmed using punched cards: the Engine was intended to use loops of Jacquard's punched cards to control a mechanical calculator, which could use as input the results of preceding computations." - https://en.wikipedia.org/wiki/Charles_Babbage#Analytical_Eng...

Toys are amazing for inspiration and innovation.

Re:This AI Boom Will Also Bust by bunderbunder on Saturday December 03, 2016 @10:25PM (#13098645)
I think your story does more to reinforce the parent post's point. The chain of events you outline spans an entire millennium. Over that time, yes, a lot of technology developed, and eventually we got to computers, and there was even the movement that gives us the term Luddite. But it was still one long chain of gradual, evolutionary developments over the course of a very long expanse of time. At no point was there anything like the kind of sudden turmoil that futurists like to claim that deep learning will produce.
Re:This AI Boom Will Also Bust by Aeolos on Monday December 05, 2016 @04:36AM (#13104337)
If the world evolved in linear terms, you would be right. However, the rate of scientific advancement has multiplied exponentially over the history of mankind: our scientific understanding has advanced more in the last 100 years than it has in the previous 10000 combined (nuclear physics, quantum physics, molecular biology, information technology, astronomy/cosmology - you name it.)

Remember, 10 years ago smartphones were in the realm of science fiction. Now, everyone has one in their pockets.

Today we can train a computer for a week to generate a cute little christmas jingle. How many years do we need to train a human child to do the same? AGI won't happen overnight, sure, but its effects of that research will be felt continuously along the way.

Re:This AI Boom Will Also Bust by murukesh_s on Sunday December 04, 2016 @06:06AM (#13099583)
We never know, that is what the original parent comment was about. Regarding gradual progression, remember - we are now in a highly connected world, where things move at tremendous pace. And a lot more population have access to the latest information. Just like the invention of the wheel, or the invention of steam engines, or the computers, transistors etc, there could be a tipping point, which could happen today as I write or tomorrow or even after a decade. We can only guess. But looking at the speed of things, I would say the probability of it happening very near is very high and is worth the hype.
Re:This AI Boom Will Also Bust by taneq on Sunday December 04, 2016 @08:41AM (#13099947)
The '40% of jobs lost' figure is reachable without any Singularity-style event. That's what you can get just by applying modern automation techniques to low hanging fruit. Self-driving vehicles, automatic checkouts at supermarkets, fully automated fast food places, robotic warehouse storage/retrieval etc.

If you used to have 10 people working at a supermarket and you now have 6 with self-checkouts, that's your 40% jobs lost (for that supermarket) right there.

Re:This AI Boom Will Also Bust by walrus1066 on Sunday December 04, 2016 @11:51AM (#13100561)
Self driving vehicles are not a low hanging fruit. We are far, far away from fully autonomous vehicles that don't need a human backup driver. This is despite massive investment by Google and others.
Re:This AI Boom Will Also Bust by taneq on Sunday December 04, 2016 @07:15PM (#13102542)
True, I shouldn't have grouped it in with self checkouts etc. When we do achieve reliable self driving capability it will be arguably the single most impressive engineering feat we've accomplished.

Tesla are describing their 'hardware 2' self driving platform, fitted to all of their new cars, as "full self-driving hardware", and they've posted several videos recently of fully autonomous journeys on public roads. They've said they plan to roll out an OTA update to enable full self driving next year.

Mercedes-Benz have a limited self-driving mode on their 2017 model.

Uber-owned Otto, a company focusing on self-driving technology for trucks, plans to start offering its services next year.

And of course Google is quietly continuing their self-driving project, although they've gone a bit quiet recently.

Now, it does look to me like we've still got a fair few corner cases to straighten out, but either all of these companies are badly wrong, or we'll be seeing self-driving vehicles on sale to the public in the next few years (even if not next year).

Re:This AI Boom Will Also Bust by logicallee on Saturday December 03, 2016 @04:14PM (#13097521)
No, I just meant to reference my discussion from there (i.e. for people to read through my comments there, after clicking.)

IOW I meant to transclude that discussion here. (Perhaps within that comment thread a good specific summary comment is: https://news.ycombinator.com/item?id=13090869)

Obviously it is hard to know when that magic moment will happen that some kind of general AI is created that can learn in some sense similarly to how humans do. My every indication and astonishment at the results that are being produced strongly suggests "at any moment". The results are absolutely astonishing every day and we have vastly more than enough firepower.

You might also be interested in this separate thread where I dealt with questions of consciousness and pain. I again reference it here:

https://news.ycombinator.com/item?id=13026020

and you can click through at the top to follow my reference.

We are way past the point of no return here, and in my estimation it is a question of years or at the most decades - not centuries.

Re:This AI Boom Will Also Bust by Noseshine on Saturday December 03, 2016 @05:13PM (#13097763)

  > Obviously it is hard to know when that magic moment will happen that some kind of general AI
  > is created that can learn in some sense similarly to how humans do. My every indication and 
  > astonishment at the results that are being produced strongly suggests "at any moment".
As an IT guy with a basic but solid neuroscience education (which isn't even needed for what I'm about to say): Yep, you are on the hype train, and very deeply. I really like reasonable discussions, I have no idea what this is right here. We will see more amazing results, sure - but applications will be specialized narrow subjects. From creating "some kind of general AI is created that can learn in some sense similarly to how humans do" we are still very far away. Your statements remind me of 1960s "future" hype,a nuclear reactor in every car by the year 2000, stuff like that.
Re:This AI Boom Will Also Bust by logicallee on Saturday December 03, 2016 @06:32PM (#13098014)
My hype is different, because in my estimation we already have the hardware. You write:

>As an IT guy with a basic but solid neuroscience education

-- could you go ahead and take a few minutes (maybe will take you 5-10) to read through my above-referenced links referencing my previous discussion and tell me whether I'm correct in your estimation on the bottom-up aspect - i.e. the amount of computation that human neural nets can likely be doing, and how it compares to server farms with fast interconnects today?

I'm not an expert in neuroscience so your feedback might be helpful there.

Re:This AI Boom Will Also Bust by xamuel on Saturday December 03, 2016 @07:35PM (#13098207)
If P=NP then we already have the hardware to crack RSA encryption.

The above sentence is true, but it has no bearing on anything.

Re:This AI Boom Will Also Bust by logicallee on Saturday December 03, 2016 @08:28PM (#13098328)
don't you think it would have a lot more bearing if you had 7 billion devices nonchallantly walking around cracking RSA every day using the same or less hardware? (but we couldn't reverse-engineer them, because they were obfuscated in biology)?

The fact that they weren't reverse-engineered (yet) would still have huge bearing on everything.

By 7 billion samples I mean the humans walking around. Your analogy with an RSA crack is fundamentally different beccause biology doesn't do it in 3 pounds of grey goo in seven billion different bodies already.

so you would have to come up with an analogy that uses something we cant use, to say, okay fine it exists and fine, we have the hardware to also do it, but the former doesn't have any bearing on us doing the latter.

Re:This AI Boom Will Also Bust by inimino on Sunday December 04, 2016 @02:18AM (#13099129)
How much computing hardware does it take to beat a human at chess or at go? (Less than you think, Deep Blue was hopelessly inefficient.) And how long did it take for those much simpler things to happen?
Re:This AI Boom Will Also Bust by xamuel on Saturday December 03, 2016 @10:41PM (#13098684)
Interesting observation. If brains routinely cracked RSA, that could be evidence that P=NP.

Still it wouldn't help us find the P-time algorithm in question. We could say "it seems to exist", but that would not imply "we'll discover it any day now".

Re:This AI Boom Will Also Bust by Noseshine on Sunday December 04, 2016 @04:30AM (#13099385)

  > If brains routinely cracked RSA
For brains numbers have a completely different meaning and internal representation than for computers. Brains don't "think in numbers". Doing the kind of math we invented is a major effort for the brain, it's not what it developed for, and it is very poorly equipped to do explicit numerical calculations (emphasis on "explicit"). So looking at brains to "crack RSA" seems like waiting for a hammer to be useful in driving screws.
Re:This AI Boom Will Also Bust by Noseshine on Sunday December 04, 2016 @03:47AM (#13099314)

  > i.e. the amount of computation that human neural nets can likely be doing
We don't know nearly enough what they are really doing! We only know a few selected bits and pieces! You are basing your assumptions on nothing, so according to logic any conclusion is possible from a faulty premise. On which you promptly deliver spectacularly.

Also, the "computation" a brain does at one moment leaves out the time aspect: Lots of things lead to constant changes. The wiring changes all the time. The "computation" metaphor has little use for describing or understanding this major aspect of "brain". The more I learned about neuroscience the more unhappy I got with the computing metaphor that I had had going in (as a CS graduate, naturally, I think). The brain is so very, very different from my pre-neuroscience-courses notions.

How much neuroscience do you know? If the answer isn't at least an undergrad introductory course (the accompanying book is over a thousand pages), why do you get the idea you can make any predictions?

Read this to read about complexity in biology vs. engineering and what scientists in the field think how well we are dealing with it:

- http://biorxiv.org/content/early/2016/05/26/055624

- http://www.cell.com/cancer-cell/fulltext/S1535-6108(02)00133...

Test yourself: Do you understand what he's talking about? http://inference-review.com/article/the-excitable-mitochondr...

Fortunately you don't have to sign up at university these days just for such knowledge:

Free courses (if you ignore the certificate nonsense):

- https://www.mcb80x.org/ (This is linked to from edX as "The Fundamentals of Neuroscience" Parts 1, 2, 3)

- https://www.coursera.org/courses?languages=en&query=neurosci... (Especially "Medical Neuroscience": https://www.coursera.org/learn/medical-neuroscience)

- https://www.edx.org/course?search_query=neuroscience

Re:This AI Boom Will Also Bust by hmijail on Tuesday December 06, 2016 @05:44AM (#13113661)
Thank you so much for those insightful links!
Re:This AI Boom Will Also Bust by lucidrains on Saturday December 03, 2016 @04:33PM (#13097605)
absolutely dude.. for example, this paper just popped up in the last week https://arxiv.org/abs/1611.02167
Re:This AI Boom Will Also Bust by nomailing on Saturday December 03, 2016 @08:47PM (#13098380)
Interesting. The concept seems to be very similar to this one: https://arxiv.org/abs/1611.01578
Re:This AI Boom Will Also Bust by lucidrains on Saturday December 03, 2016 @03:57PM (#13097462)
Re:This AI Boom Will Also Bust by xamuel on Saturday December 03, 2016 @07:24PM (#13098170)
>if there is no pixie dust that makes humans intelligent [...] then neural net breakthroughs [...] have the potential [...]

You're assuming neural nets are the right model.

Like a 19th century person saying, "if there is no pixie dust ... then eventually Newtonian mechanics will explain these unexpected wobbles we see in the planets' orbits."

Re:This AI Boom Will Also Bust by dquail on Saturday December 03, 2016 @10:43PM (#13098687)
Yes. Furthermore, neural nets are just one small part of the solution to [edit] general intelligence. A truly scalable intelligence that learns on its own, and doesn't rely on "the right answers" through training data by human experts. Without this training data, neural nets can't do much ...
Re:This AI Boom Will Also Bust by D-Coder on Sunday December 04, 2016 @04:39PM (#13101908)
I kind of doubt if neural nets have to be the right model.

Any model that gives reasonably similar results to human thinking will be useful. Planes don't have feathers and don't eat bugs, but we say that they "fly" anyway, and they carry far more cargo / passengers (and faster) than birds ever could.

Re:This AI Boom Will Also Bust by logicallee on Saturday December 03, 2016 @09:22PM (#13098484)
What do you mean "assuming" it's the right model - a known-good model is the model in human brains, which is definitely a series of neurons, no? what else would it be?
Re:This AI Boom Will Also Bust by larsiusprime on Saturday December 03, 2016 @09:58PM (#13098584)
Here's one: neurons are cells, and cells are really complex and have all sorts of internal state on different scales (organelles, molecules, etc). Neural network models in computers are an extremely simplified abstraction by comparison. And that's still ignoring all the other things that may or may not be happening in the brain beyond the connections betweens neurons themselves.
Re:This AI Boom Will Also Bust by frgtpsswrdlame on Saturday December 03, 2016 @10:14PM (#13098619)
I think you've fed majorly into the hype but to address your point there was a great comment on neurons (from someone who knows much more than I do) just a week ago: https://news.ycombinator.com/item?id=13030766
Re:This AI Boom Will Also Bust by visarga on Sunday December 04, 2016 @12:03AM (#13098863)
I read your consciousness post and will reply here because that thread is already locked. I think the problem of consciousness comes from its definition. The word consciousness is too large, encloses too many aspects of cognition, religion and philosophy to be useful. It's a suitcase word, reifying various meanings into one name.

What we need instead of "consciousness" is a set of concepts:

- "perception" - means representing inputs in such a way as to be able to adapt to the environment to achieve goals (I place qualia here)

- "judgement" - the ability to select the best action for the current state

- "reward" - comes from time to time, and is used as a signal in behavior learning

These three concepts : perception, judgement and reward are simpler, clearer and less ambiguous. They have implementations in AI, not human level yet, but getting there. These concepts make the problem concrete instead of bringing up 2000 years of attempts to "get consciousness" based off armchair philosophising.

A more interesting question than "what is consciousness?" would be: what are the reward signals that train the human brain. We do Reinforcement Learning in the brain, but the reward signal is complex, made of multiple channels, which are evolutionarily optimized for our survival. It's the engine of drive that puts us in motion. We need to reverse engineer that in order to replicate human consciousness in silico.

Consciousness is a replicator, a system that is concerned with survival and maintaining balance in the face of perturbations and entropy. What it does is to make us go find food, make babies and protect from danger - essential actions without which the human species would disappear, and consciousness with it. Or, in other words, it maximizes its rewards over time, reward being having food, shelter, company, access to learning and a few more. But it's just limited number of reward types, and they are much simpler to understand than consciousness itself.

So consciousness is just that thing that maintains itself instead of becoming disorganized and dissipating out like most physical processes that lack a self stabilizing, self replicating dynamics.

Consciousness is a balancing system for the colony of cells that just want to pass their genes into the next generation.

Re:This AI Boom Will Also Bust by logicallee on Sunday December 04, 2016 @02:12PM (#13101263)
>Consciousness is a balancing system for the colony of cells that just want to pass their genes into the next generation.

is there anything in that which requires that it be a physical biological substrate rather than a model? Can't computer models follow the same behavior as long as that's how their environment / system is set up? (More specifically, can't they also have brains made up of cells connected to each other, in a way closely analogous or even an exact simulation of chemical pathways?)

Re:This AI Boom Will Also Bust by visarga on Monday December 05, 2016 @04:28PM (#13109812)
I think there is absolutely no need for the system to be biological. A computer model can work just as well. As long as it solves the same behavior problem, there is no difference between brain reinforcement learning and artificial reinforcement learning.
Re:This AI Boom Will Also Bust by blazespin on Saturday December 03, 2016 @05:29PM (#13097810)
I agree with this. People don't understand or appreciate the potential of generative AI, especially when guided by human hands.
Re:This AI Boom Will Also Bust by dquail on Saturday December 03, 2016 @10:30PM (#13098657)
80 years is very conservative. Rich Sutton has a very well articulated argument for achieving human level hardware (using Moores law, and an estimate of the computational power of the brain derived from measurable computations in the eye (retina?).

But as you said, it's the algorithms that are by far the long pole. Current supervised learning is much akin to simple rote learning. This is the promise of reinforcement learning - and the ability to truly learn on your own through experience. That's scalable. *that said ... I'm biased being one of Rich's students :(

Re:This AI Boom Will Also Bust by mcguire on Saturday December 03, 2016 @11:05PM (#13098736)
Historically speaking, this isn't the first time someone has said any of that.
Re:This AI Boom Will Also Bust by WhitneyLand on Saturday December 03, 2016 @03:03PM (#13097259)
I hate this. Remember they did it to Michael Faraday in 1813, deriding him as not a real scientist as he was about to play a part in advancing one of the greatest challenges of mankind (unification).