That's why I fall back on trusting brand name labs. They are "staking" their reputation on a paper. If a paper turns out to be absolute BS their reputation suffers. Even if all talent is equally distributed, this makes it more likely that papers from brand name researchers and institutions are carefully reviewed because they have something to lose. This isn't right, but what is the alternative?
(7) Because most resarch happens in industry labs. Many university academics have left, or at least have dual positions in university and industry.
(2) IMO paperswithcode didn't solve much. It's nice, but just publishing code doesn't fix any of the incentive problems. You still don't know how that code was generated. Most likely by tuning random knobs until something worked.
[1] https://www.natureindex.com/news-blog/google-scholar-reveals...
[2] https://www.natureindex.com/news-blog/google-scholar-reveals...
I can't imagine two things making less tangible ("industrial") value than Mathematics and Neuroscience today. Physics is too big a field to make such an asinine comment on at least
(I also find it amusing that you categorize physics as "too big to make such an asinine comment about", but mathematics not)
My point is that the only counterexamples you could find to "industry labs > research labs" were in completely non-industrial scenarios like Mathematics and Neuroscience, two famously theoretical disciplines
Mathematics might be plenty large in theory-space, but real-world applicability is much more the domain of physics and thus I would expect many real-world applications to arise from it. So, to me, it's clear one cannot say Physics doesn't deliver on the physical from time to time
It is happening in front of our eyes. From where should we start.
(1) Responsible AI, but the outcomes didn't challenge AI research enough to stop some of the practices. I would avoid commenting on what happened at Google AI recently, but I would say that most of the AI-based solutions that are deployed at large scale can be challenged.
(2) paperswithcode.com solved this issue. Back in the day, we would re-implement models that we see in the paper. When we email authors, we will always figure that an RA implemented the solution and s/he left.
(4) Rich is becoming richer. There is a phenomena in academia to blindly trust those who are in good labs. I have seen many papers that are great coming from these labs, but there are bad papers as well. we shouldn't just blindly trust ppl.
(5) Review process is broken. Review is a voluntarily work for professors, and imho it shouldn't. In industrial R&D, reviews are one of the most important things as part of the work. Some of our PRs takes days.
(6) All recommendation engines are black-boxes. I don't know what YT or FB recommend to a kid and whether this is aligned with some of our values. If s/he accidentally watch something bad, we noticed that their timeline is full by the same ideas. In our spare time, we tried to check if we can influence this behavior as a user. I figured that all of our recommendations engines are black-boxes. I don't know why we don't challenges. I hacked an App to fetch data, new, feed and run it through recommendation engine and started customizing what I can read.
(7) It's 2021, most of good papers are coming from industrial labs.