My original point was in response to OP's saying that not every paper needs to be brilliant. The thing about "normal science" is that begins with more or less verified theories and extends their theory and practice - until they fail, reach the edge of the theories, and the science community must search for alternative hypotheses. Machine learning is more working with a bunch of practices, rules of thumb and suggestions for doing practices. These can't really fail since they're roughly repeating themselves in different domains. The problem is that can be done forever and you never need new theories or approaches.

So I'd claim we're really at the situation right now, at the exploring new ideas, the "crisis of science" phase where essentially people have to start brainstorming (and not all ideas are good here either but they need to be somewhat original).

All this is using Thomas Kuhn's Structure Of Scientific Revolutions model very roughly.

> An analogy from a couple years ago is bridge building before physics. You accumulate rules of thumb, you get a vague understand what works. You get better. But you aren't producing a systematic field.

This would be more like Kuhn's pre-paradigmatic (pre-scientific) activity, than moving from one scientific paradigm to another ?

https://samzdat.com/2018/05/19/science-under-high-modernism/

(I don't know which one is more appropriate here.)

Yeah - SVM were considered to be optimal, then they got smashed by deep networks, now people keep bringing them back and saying "they are just as good" and yet deep networks keep being used to do all the breakthrough work.

Thanks for your comment. I should go and read someone who’s actually studied the phenomenon, instead of making conjectures.