News for hackers. Stuff that matters.

A comment on Machine-Learning Maestro Michael Jordan on the Delusions of Big Data and Others.

Re:Machine-Learning Maestro Michael Jordan on the Delusions ... by fiddlediddlefoo on Tuesday October 21, 2014 @04:06AM (#8486383)
His comments are way off the mark. The recent advances in neural network training are not strictly due to convolutional neural networks, but rather the discovery that gradient descent works remarkably well on training multilayer neural networks when using modern hardware. All of the best performing pattern recognition techniques in speech, image recognition, and natural language processing now utilize "neural networks". A neural network is nothing more than a poor name for a non-linear statistical model, and if you like one with a hierarchical structure (which is made possible strictly due to the non-linearity).

I don't think that anybody in the research community (except for maybe an occasional crazy) believes that neural networks have any biological significance beyond inspiration. NIPS (Neural Information Processing Systems) has been a reputable venue for work in statistics for some years now with no confusion over the idea that "Neural" does not mean a precise (or even imprecise) imitation of biological neurons.

[ Parent | on HN ]
Re:Machine-Learning Maestro Michael Jordan on the Delusions ... by mborsuk on Tuesday October 21, 2014 @04:55AM (#8486502)
How quickly did you read this? He says very nearly what you are saying:

"Well, I want to be a little careful here. I think it’s important to distinguish two areas where the word neural is currently being used.

One of them is in deep learning. And there, each “neuron” is really a cartoon. It’s a linear-weighted sum that’s passed through a nonlinearity. Anyone in electrical engineering would recognize those kinds of nonlinear systems. Calling that a neuron is clearly, at best, a shorthand. It’s really a cartoon. There is a procedure called logistic regression in statistics that dates from the 1950s, which had nothing to do with neurons but which is exactly the same little piece of architecture."

[ Parent | on HN ]