News for hackers. Stuff that matters.
Re:Neural networks in JavaScript by dnautics on Friday July 11, 2014 @02:49PM (#8021763)
I've been playing around with neural nets and find that swarm optimization is particularly effective; you can train high-dimensional neural net spaces with less testing per iteration. I think this works because there's a lot of arbitrary symmetrical choices embedded in the neural net. For example, hidden layer neurons are essentially arbitrarily chosen.

Also, I can't use backpropagation for what I'm trying to do.

Thanks for the suggestions. I had heard of Yann LeCun, I'm trying to do image recognition in an "different" way than CNNs, I'm going to read up on the other people in the list.

Re:Neural networks in JavaScript by ihodes on Friday July 11, 2014 @05:16PM (#8022458)
I hadn't heard of PSO. Looks pretty interesting. What's your use case that you cannot use stochastic gradient descent?
Re:Neural networks in JavaScript by dnautics on Friday July 11, 2014 @07:17PM (#8023045)
I prefer using PSO versus feed-forward gradient descent because emprically for my system, a particle size of ~100 gives comparable results to standard gradient descent, on a 400-or-so neuron neural net, which is ~4x slower.

I can't use backpropagation because some of the output nodes are used to apply a transformation on the input followed by reapplication of the (now transformed) input through the neural net.

Re:Neural networks in JavaScript by ihodes on Saturday July 12, 2014 @01:38AM (#8023803)
That sounds similar to a recurrent neural net? Backprop definitely works on those with some modifications. I just asked some guys at CMU/Google, they suggested looking more at SGD or some other MCMC methods that may have included the word "expert" or maybe were similar to simulated annealing or Gibb's sampling… which I can't recall… I know nothing about it, and I'm totally unqualified to say anything about them, but maybe (hopefully!) some of those words will lead to something useful! The guys I chatted with don't study PSO, so maybe don't know enough to be helpful, but were a bit down on it in general.