To add to the injury, you wouldn't believe the extent of the actions to which some PIs become engaged in order to attain/preserve their 'power'. Illegal, unethical and pathetic.
The core problem is that PIs are human, and humans are flawed. The solution (if there is one) should take this into account, and somehow try to reduce it systematically, at an institutional level. The problem is that the ones that make the rules are not going to fight against themselves ... I wish I had something more to add, but that's it, that's the state of the field.
Actually, the problematic behaviours you describe are encouraged at the institutional level, because institutions also get more funding if they output more/better research. The fact that postdocs don't want to commit scientific suicide for the sake of morals in that context should surprise no one.
The ground truth is that scientific research has been turned into a poorly regulated industry, and we are all poorer for it.
There's a very simple solution to alleviate this pain: be much more selective starting at the undergrad level. But that generates less money, so baaaaad...
Yes, which is why I wrote,
>[...] and somehow try to reduce it systematically, at an institutional level.
This problem seems much harder to fix than research funding, because the question is how to allocate prestige and influence, not just money.
Where have you seen them?
From my experience, it is checked most of the time before it becomes fraud, though.
Condensed matter physics in a well-known university. But it does involve working with engineers and quite a lot of pub-going.
It's not cheap and you must win grants just to pay rent for your own apartment. Especially early on in your career, it's a real struggle.
Another was a well established Professor 'P' with a very large number of published papers, who continued to publish multiple papers at a high rate. There had been doubts for years due to data that seemed too cute and the fact that P always took the data and made the final figures in the paper themselves. P was finally caught out when a paper included data which could not possibly have been collected, and was outright fabricated.
It is worthing nothing that both P and X left the institute without disciplinary action beyond employment termination and signing an NDA. One of them is in another city, getting government grants and doing just fine. Everyone in the know doesn't trust any of their papers of course, but I guess that doesn't include whoever reviews their grants.
None of these are 'good people gone bad'. They are are flawed individuals with no integrity and sociopathic personality traits, and they waste a lot of decent people's time dealing with their actions.
MIT is choked full with people who've signed these sorts of NDAs. You shall not talk about wide-spread research fabrication. You shall not talk about the professors who partied on Epstein's island (except for the few honest ones who came forward; they're ostracized). You shall not talk about corrupt conflicts-of-interest. And so on.
I've heard similar things about Stanford and a few other elite schools, but I have no first-hand knowledge there.
Why is this even allowed?
I think most graduate students come in with pretty good integrity. By the time people are promoted for tenure, you've had multiple filters for sociopathic behavior.
These labs are up for renewal from HHMI every 5 years, and as such, do face the same pressure to continue publishing high-impact work.
Also, the period can increase as time goes on. Her lab doesn't have to renew for 9 years I think (they renewed last summer), if not 7.
Just my 2c, it seems to me like these funding structures do have a positive impact on the research environment. That said, it's obviously a model that really requires them to select the best of the best labs as you mention, because one org (or even the gov) quite simply can't give blank checks to every lab.
It seems like it's also a model that makes labs better. What labs will be the "best of the best labs" is not independent from the way their funding sources provide pressure/rewards/punishments to do work in certain ways or do certain kinds of work.
* I can't claim to be up-to-date on the distribution of labs funded so for all I know someone is already doing this.
What does "far from optimal" mean? Let's pretend the allocators are not god-like decision makers (if they are, let's fire all the scientists because our god-like decision makers can do the research for them).
Let's just accept that if we have a large bunch of diverse potential researchers, and limited funding, the best way to get a breakthrough is by funding a random sample of these researchers.
Maybe this sample should be stratified, but it should not be stratified in such a way that it incentives chasing grants (effectively making the grant-allocators the real PIs - with all their flaws and biases systematically driving the process, but without any accountability or management effort on their behalf).
We can't get optimal without unlimited wisdom, and if we had unlimited wisdom we wouldn't need scientists.
Counterpoint: the current system doesn't do this. It optimises for competency to attract funding. This occasionally overlaps wroth competency to perform good research, or competency to perform research well, or competency to oversee, nurture and guide research in others. But that is far from given.
The first pool is for idea generation, failure is an option. The second pool is to direct invention for social benefit.
So why not just fund them?
Any process you could create or that currently exists just ends up being gamed by people who understand how to get funding.
There's also the fact that a PI minting dozens of PhD over their careers, especially in fields with limited industry opportunities, inherently produces a pyramidal scheme that is bound to push people away from science.
We seem to have no problem with infinite funding for Department of Defense and to bail out banks. The amount needed for science is paltry by comparison and typically has much better long term return on investment.
As for why research doesn't get more funding, sadly there is a faction in the US government that is opposed to universities and to government spending in general, and views research funding as easy to cut without provoking public outcry among their core voting demographics. This is the same reason that the Department of Defense is now such a large part of research funding, that same voting demographic is far more vocally opposed to reducing military spending.
These?
"Social Security and Medicare are sometimes called "entitlements," because people meeting relevant eligibility requirements are legally entitled to benefits, although most pay taxes into these programs throughout their working lives."
https://en.wikipedia.org/wiki/Expenditures_in_the_United_Sta...
The discussion on how much money should be allocated to research is a different one, and it's always going to be finite. In the end, funds for research as conducted in universities have to be extracted from societal economic outputs, be it from taxes or otherwise, which is sadly not infinite.
And increasing funding will not increase demand in lockstep. There are indeed more people who want to do science then there is money allocated to pay them, but the number of those people is not going to vastly increase if 'researcher' continues to be a job that requires years of post-secondary education, has long, shitty hours, and pays peanuts.
There are no (steady) places for everyone, sadly. That's the way things are, where they demand postdocs to be extra productive, promising a permanent position somewhere (or a letter of recommendation, whatever).
Currently, the postdoc 'experience' has multiple names: research associate, research fellow, research engineer like it is a 'career' to pursue, with minimal benefits, salary attached to research projects (so it could stop at the end), etc.
https://forbetterscience.com/2020/03/26/chloroquine-genius-d...
However, that doesn't excuse the individuals doing it. "Because jobs" is a terrible ethical excuse.
Nevertheless, you have an interesting point there.
If it was this simple that a random person here could come up with how to "solve" academia, we'd have already done it decades ago. The ideas also lack nuance and when you get into the definitions of things (for example, p-hacking), then things become a lot more grey; are you allowed to look at a dataset that you spent 2 years collecting if your first hypothesis does not pan out? The clear cut cases are obvious to everyone, it's the grey area that takes 99% of the time to figure out.
Imagine reading a thread where everyone is proposing "solutions" to software development. It'd go something like "software development is a cesspool and 80% of it fails (see voting systems, MySpace, electronic health records, Theranos. Here's what software companies need to do:" [yes I'm being intentionally stupid to demonstrate how annoying this is]
1) stop releasing before the bugs are fixed. Software and games are rushed out. Companies need to take their time to fix the bugs so the users don't have to encounter them.
2) no more technical debt. Programmers are sloppy and introduce technical debt because they are not incentivized to do high quality programs. [yes, see how triggering that is]
3) cap team sizes. Everyone knows that large teams fail more spectacularly. Gmail and Napster and the original version of Google were made by a group of 4 people. Software teams need to be 4-5 people max.
4) programmers must use a transparency scorecard. Software companies like Oracle and IBM charge ridiculous amounts for their work. They hide costs and cut corners. Programmers should be transparent about the work they are doing each day, what data they access, and which functions they are writing.
These changes need to happen. derp derp
People here are writing comments like that funding should be tied to "how systematic, logical, and well-documented the research is" as though these things are correlated to the existing criteria at all.
Much of the criticism here is like someone who knows how to build roads criticising agile software development because it makes it look like no one knows what they are going to build in the end. It's frustrating, and wrong, but the errors are subtle and aggregative.
The thing about academia is it's full of people who love talking about ways to make it better, and rotate into positions of power where they can change things after a few years.
The solutions are complex, require convincing many different stakeholders (even if they're amenable to the change), nailing lots of detail to make it work right. Because peoples lives and careers are on the line. Reputations of entire fields, the way medical discoveries happen, billions of dollars of taxpayer money, major institutions, etc. are not things you want to hack and discover that whoops, you just incentivized the wrong thing and set back cancer research for a decade.
Because you can't possibly be saying that the current system is the best we can do, or that the problem is intractable.
(I agree with most of your proposals about software engineering btw. We should do more of this).
Sounds slow but there's thousands of such experiments happening simultaneously right now. This is how a long of major field-sized changes have happened, like the transition to conferences from journals (which had many initial problems like during tenure review or a lack of quality in reviews), etc. Ideas will lose traction at various stages (for example, there was a movement some time ago to use alpha=0.001 instead of alpha=0.05 for null hypothesis testing, which has been limited to that field or subfield).
Is the move to pre-publishing servers (like arxiv) a part of this? How does SciHub (and similar) figure into it?
It does sound slow, and a bit trivial, if I'm honest. Are there any examples you can share of successful experiments that have travelled across field boundaries?
HN does not have much actual collective expertise in this area (say: governance structures for technical research), but HN is solution-oriented, so ideas will be proposed. They just tend to be not very good. (See also: HN climate-studies threads.)
I've spent 10 or 15 minutes on this thread, and didn't read any comments actually building on what the OP said. OP's author does have expertise in this - he has been writing a series of good stories for Science magazine in this general area. Sigh.
But the article itself had that same vibe for me, especially with how cock sure it is that the 'questionable research practices' are 'fraud'. I mean, come on - someone making up the responses of a 500 people questionnaire; yes that I could call 'fraud'. But not being included as an author on a paper, or being included when you didn't contribute that much? I've been in both situations and in some cases, I was completely fine with it; in others I was a bit miffed (mostly because of the same interpersonal frictions that happen everywhere where people work together) - but in none of those I would call it anywhere near 'fraud' or even 'dishonest'. Yes, there exist people who pay 1 or 2 people to write papers for them and then publish those papers with themselves as the only author. Again, that I would call 'fraud'. But the 99.9% of other cases - not even close. Just like because there is one billionaire underage sex trafficer, doesn't mean all of them are and that 'the system' is 'broken'.
And this is how I, again, got sucked into a completely non-productive 'discussion' that is so far removed from reality so as to be completely irrelevant anyway...
As I understand it, ghosting becomes a more significant issue when it enables the omitted author to peer-review their co-authors' papers (and vice-versa) without disclosure of the conflict of interest.
Just because an idea is simple to come up with, doesn't mean it's also easy to implement.
Also, can you explain to me how your programming analogy works? Because I agree with a lot of it, though it seems I'm not supposed to.
1) stop releasing before the bugs are fixed. Software and games are rushed out. Companies need to take their time to fix the bugs so the users don't have to encounter them.
- This is a tradeoff between shipping time and bugs. You will never fix all bugs, so it's unrealistic. And it's a business decision in many cases. Even the definition of a bug is tricky, like is a usability issue a bug? So it's just a naive idea.
2) no more technical debt. Programmers are sloppy and introduce technical debt because they are not incentivized to do high quality programs.
- No one wants to create technical debt. Obviously it slows down development later on. Again this could be a business decision. Some programs like one-off data science scripts don't need to fix all their technical debt. Technical debt also accrues naturally (like just changing environment, platform, standards) so it's not possible to aim to not have debt in the very beginning. Hindsight is 20-20 and all that. Saying programmers are not incentivized to do high quality programs is just a blanket naive statement, and depends on the definition of high quality programs.
3) cap team sizes. Everyone knows that large teams fail more spectacularly. Gmail and Napster and the original version of Google were made by a group of 4 people. Software teams need to be 4-5 people max.
- Depends on the type of software. Can't just generalize given a few token examples. Expectations also change over the course of the product.
4) programmers must use a transparency scorecard. Software companies like Oracle and IBM charge ridiculous amounts for their work. They hide costs and cut corners. Programmers should be transparent about the work they are doing each day, what data they access, and which functions they are writing.
- This uses one subsection of the software economy to make a point (as a fallacy). But also some of these measures don't make sense, like some programmers read a lot of code or delete lines, and so the metrics are not generalizable.
In summary, these are ideas that someone who has not done long-term software development would say, or someone who has only had experience with one type of software or company would say. They're not well defined, not generalizable, and don't account for the complex and varied sociotechnical process that software development is.
2) Limiting the number of graduate students. The current relatively-low barrier of entry into grad school provides cheap, motivated labor for PIs who are trying to stretch their research dollars to the limit. The outcome today is far more PhD graduates than jobs, and for those who get jobs, far too many of them for the research funds that are available. The overall quality of research drops when PIs focus what is fundable, rather than what is important.
3) Shifting the burden of the bulk of research work from trainees to salaried research associates/assistants/lab techs.
4) Change the focus of research output from novelty and volume (of papers published) to quality and significance of work done. Good research is often slow and careful, and doesn't fit well with the demands of grant funding agencies.
Fewer grad students means more resources available for their training, and better career prospects. Shifting the bulk of the research benchwork to salaried professionals removes the incentive to commit fraud. None of this has to come at the expense of research quality; as an example, the NIH/HHMI already have approaches to vetting research programs for quality, even if the PIs aren't competing for grant funding.
Isn't a lot of the graduate school product providing a way for people to immigrate to the United States?
The economy around it is pretty complicated and definitely there's way more pressure to supply more, not fewer, spots.
A single professor trains dozens of grad students over the course of their career. Even if you remove every single foreigner, and every single person who does not want to pursue a career in academia, you still end up with dozens of graduates - per professor.
The number of jobs available for those graduates?
One - that professor's - when he or she retires. And until then, they get to burn the midnight oil, doing grunt work on their projects, in the hopes of competing their dozen collegaues for a shot at that one spot.
This simply isn't true. Remaining in academia is only one of many career paths for graduate students, and many are financially compensated much better.
Some examples of these include physics and math grad students being recruited by hedge funds, comp sci grad students being recruited by tech companies and geology grad students being recruited by oil, gas and resource companies.
Even within academia the size of the market isn't static. New and emerging universities are hungry for qualified research professors and many come from these programs.
Of the 'stay-in-academia' camp, the ratio is hideously stacked against the graduates.
New and emerging universities aren't doubling the academic jobs pool every five years... Which is what it would take to keep up with the graduates being churned out.
(And incidentally, they are also contributing to the oversupply, because their professors will also be churning out even more new graduates.)
Becoming a graduate student is not easy, the bar is not relatively low, and making it through the program is even harder. It's true that academic positions are not large enough but you're ignoring the private sector.
Research scientists could not possibly be paid enough by universities. These would be graduate students minus the mentorship. A worse of all worlds.
They should be tied to how systematic, logical, and well-documented the research is. We need a system wide change from funding bodies, job committees to publishing criteria.
As a researcher you should be desiring to get jobs or rewards based solely on the care of methodology, clarity of communication and ease of reproducibility.
If you do those things well and the papers turn up negative results, that’s good archived knowledge for society. Turning up positive results should be viewed as an emergent property of a wide network of labs, agencies, universities and governments, and never a property of darling individuals.
I've never seen it argued in serious research circles that grants should be given this way, but I did find the non-research community to be overly obsessed over negative results not being published or rewarded. It is simply a consequence of the set of possible experiments being infinite, the same way you can come up with an infinite number of startup ideas, but not all of them are equally good even before implementation. Should we reward startup founders with failed startups because they tried their best and really did the best they could given the circumstances?
> “ It can be argued that part of being a good researcher is developing an intuition and taste for interesting and promising research questions.”
I don’t think this can be argued actually. This is mythology, usually applied to creditmongers who run labs and accumulate accolades that are actually due to a wide array of students and post-docs who are made to get reduced credit as a type of dues paying laced with rampant discrimination and sexism.
I’d say mythologizing the idea of a crack sleuth who has a special knack for research intuition is extremely harmful on all fronts: depriving value to society because it’s false and depriving value to the wide network of lower level staff who are actually responsible for progress.
In North America at least, biomedical research labs operate largely as fiefdoms of the individual principal investigators (PIs). The actual research work falls almost entirely on the backs of grad students and postdoctoral fellows. The grad students need to generate "good" data in order to graduate, the postdocs need the same in order to gain real employment (with only about 10% gaining faculty positions themselves after many years of postdoctoral training). The PIs need such "productivity" from their trainees in order to gain the funding that keeps the labs going. The PIs themselves face success rates in grant applications that are often 10% or lower and, particularly early in their careers, their job security depends almost entirely on their ability to secure grant funding.
These competitive pressures create enormous incentives for otherwise conscientious people, all the way along the hierarchy described above, to fudge their research data. Research fraud is thus a direct outcome of a fundamentally-broken approach to the structure of research funding.
There are exceptions to that approach, however. The not-for-profit Howard Hughes Research Institute [1], and to an extent the intramural research programs of the NIH [2], offer funding for PIs to do what they do best, without the pressure of competing for scant funds. Coincidentally, some of the best science comes out of these sites.
1: https://www.hhmi.org/scientists 2: https://irp.nih.gov/about-us/what-is-the-irp