News for hackers. Stuff that matters.
Re:Visualize Algorithms based on Backpropagation by hacker42 on Wednesday June 29, 2016 @12:30PM (#12002734)
Would be interesting to see this for various test functions with various starting parameters. Here is a large compilation of various test functions: http://infinity77.net/global_optimization/genindex.html
Re:Visualize Algorithms based on Backpropagation by p1esk on Wednesday June 29, 2016 @05:13PM (#12004863)
The functions in your link are not gradient based, not sure how they are applicable to neural networks with backprop.
Re:Visualize Algorithms based on Backpropagation by hacker42 on Wednesday June 29, 2016 @05:58PM (#12005133)
Don't know what fraction of them, but most of them look differentiable (as long it is not exactly some variant of the Weierstrass function).
Re:Visualize Algorithms based on Backpropagation by p1esk on Wednesday June 29, 2016 @11:56PM (#12006455)
True, but I think these functions are intended for non-gradient based optimization algorithms.
Re:Visualize Algorithms based on Backpropagation by dpratt71 on Wednesday June 29, 2016 @07:10PM (#12005501)
Does someone want to take on the challenge of explaining the basics of this to a lay person? For starters, I don't understand the relationship between the scatter plot and the contour plot. I would guess that the blue (?) dots relate to the darker areas of the contour and the red dots to the lighter areas.
Re:Visualize Algorithms based on Backpropagation by chestervonwinch on Wednesday June 29, 2016 @08:05PM (#12005761)
There are two classes in the scatter plot -- blue and red points. These can be separated by a line. A line has two parameters, for example, a slope and an intercept (although, other parameterizations are possible). These line parameters are the x and y axis. The lighter and darker areas of the heat map indicate better and worse lines, respectively, for separating the two classes of points.
Re:Visualize Algorithms based on Backpropagation by dpratt71 on Wednesday June 29, 2016 @08:17PM (#12005806)
That makes sense, thank you.
Re:Visualize Algorithms based on Backpropagation by choosername on Wednesday June 29, 2016 @12:32PM (#12002753)
I was looking for a blogpost here that used similar, orange visualisation. The graphs seem to be an intuitively general application in geometry or analysis or whatever - I don't have the basics down; this should help.
Re:Visualize Algorithms based on Backpropagation by lormayna on Thursday June 30, 2016 @02:57AM (#12007006)
What is the advantage of using Neopy instead that others neural network Python library like Pybrain?
Re:Visualize Algorithms based on Backpropagation by itdxer on Thursday June 30, 2016 @06:17AM (#12007586)
There are a couple of advantages:

1. It's based on Theano, so it's fast. Also you are able to run your code on CPU and GPU.

2. NeuPy supports a lot of algorithms and different layer types (http://neupy.com/docs/cheatsheet.html), so you can easily construct deep neural networks.

3. In the NeuPy it's easy to read and understand network's structure. One of the newest features is a Subnetworks (http://neupy.com/docs/layers/basics.html#subnetworks).

4. Neural Network Surgery is another new feature (http://neupy.com/docs/surgery.html)