News for hackers. Stuff that matters.

A comment on Basic Neural Network on Python.

Re:Basic Neural Network on Python by theschreon on Friday July 05, 2013 @01:56PM (#5995944)
You could try the following improvements to speed up neural network training:

- Resilient Propagation (RPROP), it significantly speeds up training for full batch learning: http://davinci.fmph.uniba.sk/~uhliarik4/recognition/resource...

- RMSProp, introduced by Geoffrey Hinton, also speeds up training but can also be used for mini-batch learning: https://class.coursera.org/neuralnets-2012-001/lecture/67 (sign up to view the video)

Please consider more datasets when benchmarking methods:

- MNIST ( 70k 28x28 pixel images of handwritten digits ): http://yann.lecun.com/exdb/mnist/ . There are several wrappers for Python on github.

- UCI Machine Learning Repository: http://archive.ics.uci.edu/ml/datasets.html

[ Parent | on HN ]
Re:Basic Neural Network on Python by dfrodriguez143 on Friday July 05, 2013 @02:48PM (#5996195)
Definitely a lot to read and improvements to make. I will probably do a more complete benchmark with more datasets on a later post.

Thanks for the suggestions.

[ Parent | on HN ]
Re:Basic Neural Network on Python by benhamner on Friday July 05, 2013 @06:59PM (#5997127)
You may be interested in this ICML 2006 paper, which empirically compared many standard algorithms across a combination of metrics and UCI datasets - http://www.cs.cornell.edu/~caruana/ctp/ct.papers/caruana.icm...
[ Parent | on HN ]