News for hackers. Stuff that matters.
Re:Diffusion Models Are Evolutionary Algorithms by sevensor on Thursday November 14, 2024 @09:04AM (#42136214)
From the evolutionary algorithms perspective, the work presented in the article is tremendously ordinary, even formulaic. There’s always somebody publishing a new evolutionary search operator with a flavor of the month biological analogy. The test functions are bog standard, and although they’re designed to present certain difficulties, they do a pretty bad job of resembling actual black box engineering models; their main advantages are fast evaluation and analytically known optima. Your best bet for single objective optimization is probably still CMAES.
Re:Diffusion Models Are Evolutionary Algorithms by upghost on Thursday November 14, 2024 @04:33PM (#42141462)
Wow thanks for sharing, I had never heard of CMAES before. Very excited to play with this!
Re:Diffusion Models Are Evolutionary Algorithms by Matumio on Monday November 18, 2024 @07:06AM (#42171688)
Can confirm. I've played around with differential evolution and a few others. Nothing was able to beat CMA-ES. At best, after some manual tuning it was just barely matching the untuned CMA-ES. (The task was to optimize a small neural network, gradients not easily available.)

CMA-ES is not suited for very large problems (more than a few hundred parameters), but there are several variants that can handle this, like sep-CMA-ES.

Re:Diffusion Models Are Evolutionary Algorithms by sevensor on Monday November 18, 2024 @08:23AM (#42172089)
Differential evolution definitely comes in second as far as I’ve seen. In fact, what I’ve seen is that evolutionary search operators are effective inversely proportional to how faithfully they model actual biological processes.