Any critique of AI is fair, though. If you think PL theory is bad, try running the code from a good 90% of neurips papers. Hearing ML people complain about the reproducibility of psych research is hilarious.
Like, at least their reproducibility problem makes an iota of sense. Empirical research involving humans is inherently messy. Doesn't excuse anything, ofc, but I at least understand why the problem could be methodology rather than outright fraud. With neurips papers, I'm literally just trying to run published code using their install scripts on the exact instance of an AWS machine they said they used, and getting just fucking absolutely wildly different numbers.
Either I'm totally incompetent after 20 years of coding and publishing, or the deep learning reproducibility meltdown is going to make the psych reproducibility crisis look like a minor blip.
It's so hard for CS Researchers to write comprehensible code that does something new and actually works. I originally said "I'll never understand why", but I guess my above comment explains it perfectly: after a couple rounds of peer review, people get the picture and learn to stop writing working good explanations of working code.
Yes, a major from my many years in academia is sophistication is valued over simplicity. A bunch of fancy symbols and equations = important, so much of what is published appears, at least to me, to be much more complicated than it needs to be.
No, in that the sophistication, in the good cases, is because there is a multitude of subtletly and complexity in these subjects, as well as a formal framing of the issue so that it can be analyzed in an unambiguous manner. So, there is a reason for the obfuscation, although it mostly ingrown overkill at this point.
The best papers can both deal with the great complexity and sophisticated formalisms, while boiling it all down to an accessible delivery.
However, such papers are few and far between.
A good example is this gorgeous paper about grid-free methods for geometry processing: https://www.cs.cmu.edu/~kmcrane/Projects/MonteCarloGeometryP...
Yes, it uses greek symbols, but it's referencing mathematical concepts of geometry and the associated algebra. Hence, this is appropriate.
But that paper balances this out with illustrative diagrams and renderings. In full colour! Showing practical applications!
It's just beautiful. I wish more papers were like this one...
I get mercilessly downvoted whenever I point out that it makes zero sense to use Greek letters in computer science papers. There is no rich history of Ancient Greek computer science that is being referenced for consistency with the new research!!! It's 100% intellectual wankery, nothing more.
I experienced the same thing first-hand with AI research. Nobody is going to get published for using a straightforward numerical approach like Automatic Differentiation in a paper, so they spend 90% of their effort symbolically differentiating the formulas and end up with something that is numerically unstable but oh so pretty and impressive looking when laid out with LaTeX.
Publishing in science is now all about showing off, and not at all about the advancement of science. It's politics, not practice.
PS: Go back and compare CS papers published in the 60s, 70s, and even 80s with stuff being put out now. It's night & day!