News for hackers. Stuff that matters.

A comment on What does a neural network actually do?.

Re:What does a neural network actually do? by csense on Sunday May 25, 2014 @04:56PM (#7797538)
I've found that, in practice, traditional neural networks tend to be prone to overfitting and are finicky about their parameters (in particular the topology and number of nodes you choose).

I use the word "traditional" to describe the NN architecture discussed in the article. Recent NN research has been promising [1], but this article strictly discusses traditional NN's. I don't really have much experience with the newer NN algorithms, so I'm not sure to what extent they suffer from the same problems as traditional NN's.

[1] http://en.wikipedia.org/wiki/Neural_network#Recent_improveme...

[ Parent | on HN ]
Re:What does a neural network actually do? by billderose on Sunday May 25, 2014 @06:54PM (#7797909)
Hinton's DropOut [1] and Wan's DropConnect [2] have ameliorated some of the overfitting issues present in traditional NN's. In fact, DropConnect in conjunction with deep learning are responsible for new records being set on classical datasets such as MNIST.

[1] http://arxiv.org/pdf/1207.0580.pdf [2] http://cs.nyu.edu/~wanli/dropc/

[ Parent | on HN ]
Re:What does a neural network actually do? by agibsonccc on Sunday May 25, 2014 @09:18PM (#7798281)
Dropout is actually a knob on any neural network. These are used in image recognition as well as text and other areas.

The fuzzing creates a very similar effect to convolutional nets where it can learn different poses of an image.

[ Parent | on HN ]
Re:What does a neural network actually do? by im3w1l on Sunday May 25, 2014 @08:15PM (#7798135)
It's pretty funny, I saw DropConnect described in a stackoverflow answer that predated the paper you reference. It was an incorrect answer on how to do dropout. I shall try to find it tomorrow.
[ Parent | on HN ]
Re:What does a neural network actually do? by niels_olson on Sunday May 25, 2014 @06:19PM (#7797808)
Is it safe to say that in ML, use of NNs is more about writing code that designs NNs, evaluates the results, and modifies the designs to optimize some desired meta-values, like accuracy, efficiency, etc?
[ Parent | on HN ]