Re:Empiricism and the limits of gradient descent
by ssivark on Sunday May 27, 2018 @09:38PM
(#17170144)
I don't see why stochastic has to be worse than evolutionary algorithms. In high-dimensional spaces, there is a large space of possible "mutations". SGD just biases certain mutations based on the gradient from a minibatch. That sounds a lot like evaluating the fitness of a mutated population on a minibatch and culling members with low fitness. In fact, there are many demonstrations that the stochastic nature of SGD (coming from the use of minibatches) is crucial for effective learning.
[ Parent
| on HN ]
There are no comments on this comment yet.