And every other comment was like: “What did he expect, he had much less than the usual two papers a year.”
So it seems that even here on HN, the mindset of quantity over quality still persists.
In the long term, it seems like journals that publish lots of falsified papers should be punished, and journals that don't (e.g. because of a judge-upon-pre-registration policy) should crowd them out.
Though this will inevitably lead to the problem that grant boards face, that it'll be a lot harder to differentiate between proposed experiments. And it'll be even harder to do boring stuff. (So if we assume all submitted plans are sound, they have to publish them all. Though then we'll have journals based on how strict they are with experiment design requirements, 1 sigma, 2 sigma, 5 sigma, etc.)
From their 2019 website[0]: "Do you have insightful and exciting work sitting in a drawer somewhere because it never quite panned out? Are you willing to share your failed approaches so that others can learn from them without having to re-travel the same road? Are you tired of reading papers that pretend the incremental result they happened to achieve was well-motivated and was their goal all along?
We are! That’s why we are founding a new conference: a place for papers that describe instructive failures or not-yet-successes, as they may prefer to be called."
PLOS e.g. specifically stated multiple times that they'll publish what meets their quality standards regardless of a positive or negative outcome.
The problem is: Even if you publish failed research it won't get cited as much. And people still use citation metrics to evaluate "quality" of science.
Just having journals that publish your "failed" research is a good start, but it's not changing the incentive structure.
The greatest thinkers in history had no pressure outside their own urgence to solve/explore. Surely, not everyone is Galileo but today science need that kind of freedom.
[0] https://fads.ws/
There are an infinite set of configurations that could fail, and it's not sufficiently useful to know they failed without understanding why they failed. And analysing failures in a useful way is an extremely difficult and fundamental problem. On the other hand, a successful experiment is an extremely rare event and hence interesting by itself.
Getting data is always expensive in any field so it's much easier to analyse data that already exists as you don't need a large grant application.
Furthermore, I think the use of certain ML techniques may be akin to resume-driven development particularly for PhD students given that the career prospects in Data Science in industry (and using the HR buzzwords like AI, ML, Deep Learning etc.) are much better than the thin pickings that remain in academia in many fields.
I did the same thing as a physics student, but with electronics and programming, which furnished me with a marketable resume.