News for hackers. Stuff that matters.
Re:Techniques for Training Large Neural Networks by erwincoumans on Thursday June 09, 2022 @04:25PM (#31686331)
Black box optimization such as CMA-ES or Augmented Random Search (ars) are used by researchers, for example:

CMA-ES: https://blog.otoro.net/2017/10/29/visual-evolution-strategie...

ARS: https://arxiv.org/abs/1803.07055

Those optimizers are efficient up to certain neural network sizes.

There are no comments on this comment yet.