News for hackers. Stuff that matters.
Re:String theory lied to us and now science communication is... by godelski on Monday November 13, 2023 @02:18AM (#38247802)
One thing I've been trying to push is the idea of Goodhart's Hell and how we're all metric hacking. In academia, one of these things is fundamentally about how papers are written. As someone finishing up my PhD I can't wait to graduate and do real fucking research. The difference? I am not writing papers to my colleagues and fellow researchers in my niche; I'm writing papers to reviewers. These may seem like the same thing but they are very different. In a similar vein, many researchers are chasing hype and publicity. If you don't you don't get through these dumb filters. Journals and conferences do nothing (well at least conferences serve as a meetup, but not more than that). All they exist for, in modern times, is to give a signal to the bureaucrats at universities designed to operate as businesses to judge if a researcher is a good researcher or not (to be clear, this signal doesn't actually indicate such a concept, which is near impossible to tell). This, and other factors, underlines the whole replication crisis, hype (dear fucking god ML...(my area of study!)), and all this bullshit.

Research papers are about scientists communicating with one another. It doesn't matter if it is a preprint like arxiv, Nature, CVPR, or a fucking blog, these are all scientists communicating with one another. I can't wait to be out to just take my time with research, post to OpenReview and my blog, and move on not wasting my time explaining to a reviewer why metric we reproduced is different from the one in the original paper despite already including references of other works who also could not reproduce the original number and getting rejected because I had a broken link to my appendix. It's a fucking waste of time and it is a complete embarrassment that we call this process science.

There are no comments on this comment yet.