No, the comments are not confused. I wouldn't consider myself as a expert, but I've worked a lot with evolutionary algorithms. This is not a good work and should be rejected from ICLR, although again it is my personal opinion. Terminologies are there to help distinguish between different concepts, not to be obscured in some handwavy way to seem deep and knowledgable to sucker some ICLR paper reviewer with no scientific rigor. It is indeed possible to optimize NNs using GAs but claiming diffusion is an evolutionary algorithm is a huge stretch, and they even didn't draw the connection properly in the paper. There might be something worthwhile in their work but I would say portraying it this way is just flat-out not correct.

I think we are agreeing. I'm not claiming the comments are incorrect in the regard you are mentioning, I mean to say that the paper is conflating genetic algorithms (GAS) as tho they are the only sort of EA.

Words mean things and my goal is to enhance the discourse by clarifying the imprecise language used in the paper regarding the term "Evolutionary Algorithm".

I make no comment on any other issues with the paper.

This does seem like a paper worth publishing but I agree it doesn't belong in a conference like ICLR.

From the evolutionary algorithms perspective, the work presented in the article is tremendously ordinary, even formulaic. There’s always somebody publishing a new evolutionary search operator with a flavor of the month biological analogy. The test functions are bog standard, and although they’re designed to present certain difficulties, they do a pretty bad job of resembling actual black box engineering models; their main advantages are fast evaluation and analytically known optima. Your best bet for single objective optimization is probably still CMAES.

Wow thanks for sharing, I had never heard of CMAES before. Very excited to play with this!

Can confirm. I've played around with differential evolution and a few others. Nothing was able to beat CMA-ES. At best, after some manual tuning it was just barely matching the untuned CMA-ES. (The task was to optimize a small neural network, gradients not easily available.)

CMA-ES is not suited for very large problems (more than a few hundred parameters), but there are several variants that can handle this, like sep-CMA-ES.

Differential evolution definitely comes in second as far as I’ve seen. In fact, what I’ve seen is that evolutionary search operators are effective inversely proportional to how faithfully they model actual biological processes.