News for hackers. Stuff that matters.
Re:The truth about deep learning by clmcleod on Wednesday June 01, 2016 @10:08PM (#11819878)
The universal approximator approach is another big one that people lean on from the opposite direction (I suppose you could call it the "anti-deep learning" school of thought). Most people don't take the time to consider what a universal approximator actually is.

Are humans universal approximators? Absolutely not. The mere proof that a nn with an infinite number of hidden nodes (which isn't practical, which could end the discussion there) can find a statistical mapping between any two input and outputs is hardly a benefit. There is something so much more to intelligence --- the ability to generalize well (aka not overfitting) is first and foremost.

I'd push you to ask why you perceive that stacking RBMs produces "eerily human-like" results. I'd argue that they don't: http://blog.keras.io/how-convolutional-neural-networks-see-t...

Re:The truth about deep learning by return0 on Wednesday June 01, 2016 @10:22PM (#11819927)
Trying to compare anything to "intelligence" is a trap, imho. Intelligence is not a natural quantity, it's a word. There was no reason for a brain to become a universal approximator, or to have any specific mathematical property for that matter. Brains evolved to adapt behavior to the environment using the senses. The universe didn't care for creating some property called "intelligence".

I do see many results in NLP for example that have an eerie human-likeness, even if nobody can explain why. I am not an expert in the field, but i would think it's a challenge to find similar results made using another technique.

Re:The truth about deep learning by cubano on Wednesday June 01, 2016 @11:29PM (#11820170)
> The universe didn't care for creating some property called "intelligence".

On the face of it, I would have to disagree. It would seem to me that's exactly what the universe selected for, among many other things of course, in our tiny little corner of it.

Otherwise, we wouldn't be here debating it.

Re:The truth about deep learning by kmiroslav on Thursday June 02, 2016 @12:23AM (#11820350)
Maybe this universe did select for intelligence.

How about all the other universes that never generated intelligence?

But going back to our own universe, how widespread is intelligence? It's pretty much microscopic, isn't it? I think our universe seems to be mostly geared toward generating empty space, and the rare areas that are not empty are certainly extremely hostile to intelligence.

Re:The truth about deep learning by ionforce on Thursday June 02, 2016 @09:05AM (#11822087)
What do you mean hostile to intelligence? Did you mean hostile to life? And then life being a requirement for intelligence?
Re:The truth about deep learning by kmiroslav on Thursday June 02, 2016 @09:27AM (#11822273)
Yes. If life can't arise, neither can intelligence.

Our universe is massively hostile to both life and intelligence.

Re:The truth about deep learning by return0 on Wednesday June 01, 2016 @11:35PM (#11820186)
Debating the anthropic principle will be left for another thread.
Re:The truth about deep learning by visarga on Thursday June 02, 2016 @12:38AM (#11820405)
Predicting (function approximation) is perhaps the main use of neural nets but there is another very important use: generating, or in other words, imagination. And in generative mode we need to know about probabilities and latent variables - thus, we need a little more than neural nets to do this task.