News for hackers. Stuff that matters.
Re:Cracking down on research fraud by gwerbret on Sunday July 26, 2020 @08:25PM (#23961226)
These issues of research fraud come up often, but the root of the problem is a bit more subtle.

In North America at least, biomedical research labs operate largely as fiefdoms of the individual principal investigators (PIs). The actual research work falls almost entirely on the backs of grad students and postdoctoral fellows. The grad students need to generate "good" data in order to graduate, the postdocs need the same in order to gain real employment (with only about 10% gaining faculty positions themselves after many years of postdoctoral training). The PIs need such "productivity" from their trainees in order to gain the funding that keeps the labs going. The PIs themselves face success rates in grant applications that are often 10% or lower and, particularly early in their careers, their job security depends almost entirely on their ability to secure grant funding.

These competitive pressures create enormous incentives for otherwise conscientious people, all the way along the hierarchy described above, to fudge their research data. Research fraud is thus a direct outcome of a fundamentally-broken approach to the structure of research funding.

There are exceptions to that approach, however. The not-for-profit Howard Hughes Research Institute [1], and to an extent the intramural research programs of the NIH [2], offer funding for PIs to do what they do best, without the pressure of competing for scant funds. Coincidentally, some of the best science comes out of these sites.

1: https://www.hhmi.org/scientists 2: https://irp.nih.gov/about-us/what-is-the-irp

Re:Cracking down on research fraud by moralestapia on Sunday July 26, 2020 @09:58PM (#23961659)
>labs operate largely as fiefdoms of the individual principal investigators

To add to the injury, you wouldn't believe the extent of the actions to which some PIs become engaged in order to attain/preserve their 'power'. Illegal, unethical and pathetic.

The core problem is that PIs are human, and humans are flawed. The solution (if there is one) should take this into account, and somehow try to reduce it systematically, at an institutional level. The problem is that the ones that make the rules are not going to fight against themselves ... I wish I had something more to add, but that's it, that's the state of the field.

Re:Cracking down on research fraud by rscho on Monday July 27, 2020 @03:38AM (#23962971)
> The core problem is that PIs are human

Actually, the problematic behaviours you describe are encouraged at the institutional level, because institutions also get more funding if they output more/better research. The fact that postdocs don't want to commit scientific suicide for the sake of morals in that context should surprise no one.

The ground truth is that scientific research has been turned into a poorly regulated industry, and we are all poorer for it.

There's a very simple solution to alleviate this pain: be much more selective starting at the undergrad level. But that generates less money, so baaaaad...

Re:Cracking down on research fraud by moralestapia on Monday July 27, 2020 @12:43PM (#23966789)
>[...] the problematic behaviours you describe are encouraged at the institutional level

Yes, which is why I wrote,

>[...] and somehow try to reduce it systematically, at an institutional level.

Re:Cracking down on research fraud by rapjr9 on Monday July 27, 2020 @07:58PM (#23970764)
This is also why you see announcements of new "breakthroughs" and then nothing further ever happens. The experience is so unpleasant for many people (often compared to slavery) that they leave when they get their degree and never work on that project again. The PI's also get bored and want to do something new to keep building their reputation (so onward to the next grad student or post doc). Patents are sometimes generated, but new ideas are very difficult to take to market successfully so those that can afford to buy or license them do so very carefully (unless they want the patent to attack some competitors product.) In general there is rarely any follow through on new ideas, unless you work on projects with DARPA funding where they insist on "technology transfer", and even then many ideas turn out to be unworkable because they require additional extensive engineering for the practical aspects.
Re:Cracking down on research fraud by canjobear on Sunday July 26, 2020 @09:29PM (#23961534)
Another variant of the incentive problem is more insidious. Many academics are driven by ego, and specifically a desire to be influential. One way to be influential is to have great, original, and correct ideas. This can be a good incentive, but coupled with our human ability to deceive ourselves, it can become pathological. I’ve seen researchers get convinced that their ideas are so beautiful and right-feeling that they just can’t be wrong, and they will torture the data and run experiment after experiment it appears to work. Sometimes this crosses into outright fraud without the researcher even realizing it——after all, the theory is right, so if the data don’t match it they must be wrong!

This problem seems much harder to fix than research funding, because the question is how to allocate prestige and influence, not just money.

Re:Cracking down on research fraud by prionassembly on Sunday July 26, 2020 @11:01PM (#23961890)
> I’ve seen researchers get convinced

Where have you seen them?

Re:Cracking down on research fraud by kergonath on Monday July 27, 2020 @02:13AM (#23962615)
This confirmation bias is everywhere. As in, “this calculation looks off, we should re-run it” or “this number is wrong, let’s call it an outlier”. Sometimes there’s a good reason, sometimes it’s wishful thinking and they are hoping the next one will be more “correct”. Distinguishing the two is not always easy, either.

From my experience, it is checked most of the time before it becomes fraud, though.

Re:Cracking down on research fraud by prionassembly on Tuesday July 28, 2020 @02:06PM (#23978823)
Yes, but where is your experience? Academia, industry, STEM, biosciences, the pub?
Re:Cracking down on research fraud by kergonath on Wednesday July 29, 2020 @01:56PM (#23989661)
All of them except biosciences :)

Condensed matter physics in a well-known university. But it does involve working with engineers and quite a lot of pub-going.

Re:Cracking down on research fraud by Balgair on Monday July 27, 2020 @03:30PM (#23968539)
Aside: To give readers an idea of the financial pressures I have an example. One of my former professors told me that the lab spaces costs ~$45/sqft/mo from the university, 300sqft minimum. That's ~$160k/year just for the floor space in just the lab, not the office space either. Now, electrical, heat, vivarium, elevators, janitors, security, etc. are all rolled up into that. Bit that's not specific equipment costs, reagents, grad students, and the profssor's own salary. Granted, every university is different (just look at patent ownership clauses).

It's not cheap and you must win grants just to pay rent for your own apartment. Especially early on in your career, it's a real struggle.

Re:Cracking down on research fraud by Gatsky on Sunday July 26, 2020 @08:50PM (#23961345)
I have to disagree with this 'good people gone bad' analysis. None of the research fraud I have witnessed fits this model.
Re:Cracking down on research fraud by caterama on Sunday July 26, 2020 @08:57PM (#23961381)
Could you give more details on other causes of research fraud you've seen?
Re:Cracking down on research fraud by Gatsky on Sunday July 26, 2020 @09:13PM (#23961442)
One was a post-doctoral scientist, called X. The lab I was in was asked to take X on by the institute because they had a falling out with another lab, the details of which were not disclosed. A year later, after very poor productivity and various petty issues with other people in the lab, someone checked X's data collection where the original data was available on the instrtument, and found that it was all made up. Subsequently the previous lab investigated, and a paper X published as first author was retracted. Turns out in the previous lab there were also concerns about data fabrication and many other problematic interactions before the fraud was discovered. Also turns out X left another lab before this under similar circumstances.

Another was a well established Professor 'P' with a very large number of published papers, who continued to publish multiple papers at a high rate. There had been doubts for years due to data that seemed too cute and the fact that P always took the data and made the final figures in the paper themselves. P was finally caught out when a paper included data which could not possibly have been collected, and was outright fabricated.

It is worthing nothing that both P and X left the institute without disciplinary action beyond employment termination and signing an NDA. One of them is in another city, getting government grants and doing just fine. Everyone in the know doesn't trust any of their papers of course, but I guess that doesn't include whoever reviews their grants.

None of these are 'good people gone bad'. They are are flawed individuals with no integrity and sociopathic personality traits, and they waste a lot of decent people's time dealing with their actions.

Re:Cracking down on research fraud by woofie11 on Monday July 27, 2020 @07:30AM (#23964071)
That's the damnable thing. The result in each case is an NDA. These things never go public. It's in no one's interest.

MIT is choked full with people who've signed these sorts of NDAs. You shall not talk about wide-spread research fabrication. You shall not talk about the professors who partied on Epstein's island (except for the few honest ones who came forward; they're ostracized). You shall not talk about corrupt conflicts-of-interest. And so on.

I've heard similar things about Stanford and a few other elite schools, but I have no first-hand knowledge there.

Why is this even allowed?

Re:Cracking down on research fraud by gwerbret on Sunday July 26, 2020 @09:18PM (#23961477)
I agree completely that some people fit that bill; I'd argue however that even for them, the skewed incentives of research success certainly help support that particular approach to fraud. My premise was that most of the people committing research fraud are probably not sociopaths.
Re:Cracking down on research fraud by woofie11 on Monday July 27, 2020 @07:27AM (#23964046)
The bigger problem is that research fraud leads to research impact which leads to academic hiring.

I think most graduate students come in with pretty good integrity. By the time people are promoted for tenure, you've had multiple filters for sociopathic behavior.

Re:Cracking down on research fraud by etrautmann on Monday July 27, 2020 @12:19AM (#23962206)
HHMI does fund some of the most enormously successful work in biomedical science, but they start with the very best of the best. It's not clear how much of these labs success is the result of the precise funding structure, although it's obviously helpful to spend less time writing grants (or just have additional resources).

These labs are up for renewal from HHMI every 5 years, and as such, do face the same pressure to continue publishing high-impact work.

Re:Cracking down on research fraud by obmelvin on Monday July 27, 2020 @01:11AM (#23962399)
HHMI is crazy, it has definitely had a positive impact on my girlfriends PhD experience and allowed her to take on a 'post doc killer' project. It's taken her almost 3 years, but she's gotten great quality data, and just got a great review back from Nature Chem with proposed experiments as expected.

Also, the period can increase as time goes on. Her lab doesn't have to renew for 9 years I think (they renewed last summer), if not 7.

Just my 2c, it seems to me like these funding structures do have a positive impact on the research environment. That said, it's obviously a model that really requires them to select the best of the best labs as you mention, because one org (or even the gov) quite simply can't give blank checks to every lab.

Re:Cracking down on research fraud by jrochkind1 on Monday July 27, 2020 @08:58AM (#23964614)
> That said, it's obviously a model that really requires them to select the best of the best labs as you mention...

It seems like it's also a model that makes labs better. What labs will be the "best of the best labs" is not independent from the way their funding sources provide pressure/rewards/punishments to do work in certain ways or do certain kinds of work.

Re:Cracking down on research fraud by obmelvin on Monday July 27, 2020 @10:30AM (#23965397)
Yes, you are very much correct. Unfortunately, if HHMI and the like are unwilling* to do these sorts of funding experiments, I'm not really sure who will.

* I can't claim to be up-to-date on the distribution of labs funded so for all I know someone is already doing this.

Re:Cracking down on research fraud by etrautmann on Monday July 27, 2020 @11:42AM (#23966166)
HHMI is not the only one - certainly DARPA is willing to provide large funds for specific areas of interest (though a VERY different funding model). Wellcome trust, the Allen Institute, Broad institute, and others also work with slightly different models that can complement the NIH funding lines.
Re:Cracking down on research fraud by dasudasu on Sunday July 26, 2020 @08:43PM (#23961318)
The topic of funding is a recurrent one in these types of discussions. There are always more people wanting to do science than what is available in funding. There needs to be a process to allocate resources that takes into account competency for the job. Just granting it at random or equally is probably very very far from optimal if it means that an exceptional PI cannot even do the things to push his/her research to the next level. Research funding is a bit like a command economy in that there are no market forces that can be drawn upon to just sort it naturally, so of course it ends up being especially thorny.
Re:Cracking down on research fraud by wisty on Sunday July 26, 2020 @10:51PM (#23961855)
> Just granting it at random or equally is probably very very far from optimal if it means that an exceptional PI cannot even do the things to push his/her research to the next level.

What does "far from optimal" mean? Let's pretend the allocators are not god-like decision makers (if they are, let's fire all the scientists because our god-like decision makers can do the research for them).

Let's just accept that if we have a large bunch of diverse potential researchers, and limited funding, the best way to get a breakthrough is by funding a random sample of these researchers.

Maybe this sample should be stratified, but it should not be stratified in such a way that it incentives chasing grants (effectively making the grant-allocators the real PIs - with all their flaws and biases systematically driving the process, but without any accountability or management effort on their behalf).

We can't get optimal without unlimited wisdom, and if we had unlimited wisdom we wouldn't need scientists.

Re:Cracking down on research fraud by TomMarius on Monday July 27, 2020 @01:43AM (#23962510)
Why do you need unlimited wisdom to rate the merits of the people?
Re:Cracking down on research fraud by Beldin on Monday July 27, 2020 @01:59AM (#23962567)
> There needs to be a process to allocate resources that takes into account competency for the job.

Counterpoint: the current system doesn't do this. It optimises for competency to attract funding. This occasionally overlaps wroth competency to perform good research, or competency to perform research well, or competency to oversee, nurture and guide research in others. But that is far from given.

Re:Cracking down on research fraud by voxl on Sunday July 26, 2020 @10:58PM (#23961882)
I think there should be two pools of money, one that is granted at random to anyone who has obtained a professorship at a university, let that be the first filter for competency. The second pool of money can be given to projects that have immediate social interest and impact, and the government can, in this case, be very selective as there should be some social weight to succeeding here.

The first pool is for idea generation, failure is an option. The second pool is to direct invention for social benefit.

Re:Cracking down on research fraud by auggierose on Monday July 27, 2020 @12:13AM (#23962186)
I guess the first positive outcome of this would be that everybody working at a university would be immediately promoted to be a professor. ;-)
Re:Cracking down on research fraud by jasonhong on Sunday July 26, 2020 @10:27PM (#23961781)
To followup, I believe there's a subtle problem with funding in general, which is that increasing the funding available means that more research positions will be created, which means that eventually grant proposals go back to the same dismal funding rates they were before.
Re:Cracking down on research fraud by ivalm on Monday July 27, 2020 @01:38AM (#23962488)
That's an interesting Malthusian argument. I think I mostly agree in a sense that vast majority of PhD students I know had plans of staying in academia but had to join industry precisely because lack of resources. But if there were more resources and they did join academia they would then oversee an even greater number of grad students until we again hit subsistence.
Re:Cracking down on research fraud by asciimov on Sunday July 26, 2020 @09:07PM (#23961419)
> There are always more people wanting to do science than what is available in funding.

So why not just fund them?

Any process you could create or that currently exists just ends up being gamed by people who understand how to get funding.

Re:Cracking down on research fraud by dasudasu on Sunday July 26, 2020 @09:26PM (#23961519)
There are no ways around that. You cannot feasibly make funding infinite nor do anything about the lack of political will for increasing research funding. Even if funding was to increase dramatically, you'd get people coming up with even more expensive research projects (new particle accelerator anyone?) and there would still be winners and losers of that funding game.

There's also the fact that a PI minting dozens of PhD over their careers, especially in fields with limited industry opportunities, inherently produces a pyramidal scheme that is bound to push people away from science.

Re:Cracking down on research fraud by jgeada on Sunday July 26, 2020 @09:43PM (#23961589)
Why not?

We seem to have no problem with infinite funding for Department of Defense and to bail out banks. The amount needed for science is paltry by comparison and typically has much better long term return on investment.

Re:Cracking down on research fraud by dodobirdlord on Sunday July 26, 2020 @10:14PM (#23961727)
It would be far more fair to say that we have no problem with infinite funding for non-discretionary entitlement spending, since that's most of that the Federal Government spends money on. Military spending in the US is not wildly outside of the norm for the developed world as a percentage of GDP, and money spent to "bail out banks" is a rounding error in terms of spending, and has historically turned a profit for the Federal Government. One of the largest sources of research funding in the US is actually the Department of Defense.

As for why research doesn't get more funding, sadly there is a faction in the US government that is opposed to universities and to government spending in general, and views research funding as easy to cut without provoking public outcry among their core voting demographics. This is the same reason that the Department of Defense is now such a large part of research funding, that same voting demographic is far more vocally opposed to reducing military spending.

Re:Cracking down on research fraud by adamsea on Monday July 27, 2020 @12:01AM (#23962139)
What is entitlement spending?

These?

"Social Security and Medicare are sometimes called "entitlements," because people meeting relevant eligibility requirements are legally entitled to benefits, although most pay taxes into these programs throughout their working lives."

https://en.wikipedia.org/wiki/Expenditures_in_the_United_Sta...

Re:Cracking down on research fraud by dodobirdlord on Monday July 27, 2020 @03:29PM (#23968534)
Pretty much, “entitlement” isn’t a technical term but it refers to government spending on pre-defined benefit programs. A better term above might have been Non-discretionary Spending, but most of it is entitlement spending. And yes, Social Security and Medicare are the two most significant components, followed by Medicaid as the third. Those three plus interest on debt constitute the majority of the federal budget.
Re:Cracking down on research fraud by adamsea on Tuesday July 28, 2020 @11:10AM (#23976595)
For sure. I was just trying to be sneaky and point out that we pay taxes for our "entitlements" so even though it's an accepted term, I think it's an inaccurate one and implies things about the programs people call "entitlement" programs which aren't true.
Re:Cracking down on research fraud by dasudasu on Sunday July 26, 2020 @10:01PM (#23961668)
The funding for the military is far from infinite, and the US is actually pretty "privileged" as far as how much money trickles down to grants for seemingly unrelated stuff through DARPA and other similar agencies.

The discussion on how much money should be allocated to research is a different one, and it's always going to be finite. In the end, funds for research as conducted in universities have to be extracted from societal economic outputs, be it from taxes or otherwise, which is sadly not infinite.

Re:Cracking down on research fraud by vkou on Sunday July 26, 2020 @09:53PM (#23961633)
You don't need to make funding infinite, in order to meet the current demand for it.

And increasing funding will not increase demand in lockstep. There are indeed more people who want to do science then there is money allocated to pay them, but the number of those people is not going to vastly increase if 'researcher' continues to be a job that requires years of post-secondary education, has long, shitty hours, and pays peanuts.

Re:Cracking down on research fraud by tridentlead on Sunday July 26, 2020 @10:40PM (#23961824)
Well if the goal is to fund everyone, the pressures that ensure it “ requires years of post-secondary education, has long, shitty hours, and pays peanuts” will likely reduce.
Re:Cracking down on research fraud by raister on Monday July 27, 2020 @02:02AM (#23962575)
> with only about 10% gaining faculty positions themselves after many years of postdoctoral training

There are no (steady) places for everyone, sadly. That's the way things are, where they demand postdocs to be extra productive, promising a permanent position somewhere (or a letter of recommendation, whatever).

Currently, the postdoc 'experience' has multiple names: research associate, research fellow, research engineer like it is a 'career' to pursue, with minimal benefits, salary attached to research projects (so it could stop at the end), etc.

Re:Cracking down on research fraud by hhas01 on Monday July 27, 2020 @04:50AM (#23963326)
Rest assured it’s not just North America.

https://forbetterscience.com/2020/03/26/chloroquine-genius-d...

Re:Cracking down on research fraud by henriquemaia on Monday July 27, 2020 @02:22AM (#23962654)
I agree with the subtle point behind the fraud crisis: misaligned incentives.

However, that doesn't excuse the individuals doing it. "Because jobs" is a terrible ethical excuse.

Nevertheless, you have an interesting point there.

Re:Cracking down on research fraud by martincmartin on Sunday July 26, 2020 @08:37PM (#23961288)
What do you suggest as an alternative?
Re:Cracking down on research fraud by adminprof on Sunday July 26, 2020 @11:16PM (#23961955)
This entire thread is really frustrating for me to read. A lot of these ideas are one-liners from people who have had very little experience with different parts of academia, except maybe being a student. Every single idea here are debated to death by academics already (not just university academics, but funding agencies, academic societies, award panels, journal editors and conference organizers), with changes happening all the time.

If it was this simple that a random person here could come up with how to "solve" academia, we'd have already done it decades ago. The ideas also lack nuance and when you get into the definitions of things (for example, p-hacking), then things become a lot more grey; are you allowed to look at a dataset that you spent 2 years collecting if your first hypothesis does not pan out? The clear cut cases are obvious to everyone, it's the grey area that takes 99% of the time to figure out.

Imagine reading a thread where everyone is proposing "solutions" to software development. It'd go something like "software development is a cesspool and 80% of it fails (see voting systems, MySpace, electronic health records, Theranos. Here's what software companies need to do:" [yes I'm being intentionally stupid to demonstrate how annoying this is]

1) stop releasing before the bugs are fixed. Software and games are rushed out. Companies need to take their time to fix the bugs so the users don't have to encounter them.

2) no more technical debt. Programmers are sloppy and introduce technical debt because they are not incentivized to do high quality programs. [yes, see how triggering that is]

3) cap team sizes. Everyone knows that large teams fail more spectacularly. Gmail and Napster and the original version of Google were made by a group of 4 people. Software teams need to be 4-5 people max.

4) programmers must use a transparency scorecard. Software companies like Oracle and IBM charge ridiculous amounts for their work. They hide costs and cut corners. Programmers should be transparent about the work they are doing each day, what data they access, and which functions they are writing.

These changes need to happen. derp derp

Re:Cracking down on research fraud by nl on Sunday July 26, 2020 @11:51PM (#23962092)
Not sure who downvoted this, but this is 100% on the money in my experience.

People here are writing comments like that funding should be tied to "how systematic, logical, and well-documented the research is" as though these things are correlated to the existing criteria at all.

Much of the criticism here is like someone who knows how to build roads criticising agile software development because it makes it look like no one knows what they are going to build in the end. It's frustrating, and wrong, but the errors are subtle and aggregative.

Re:Cracking down on research fraud by adminprof on Monday July 27, 2020 @02:27AM (#23962676)
Yeah I have no idea how it got to -2, but maybe it was my tone. I was even trying to not call out any specific post or debate any single issue.

The thing about academia is it's full of people who love talking about ways to make it better, and rotate into positions of power where they can change things after a few years.

The solutions are complex, require convincing many different stakeholders (even if they're amenable to the change), nailing lots of detail to make it work right. Because peoples lives and careers are on the line. Reputations of entire fields, the way medical discoveries happen, billions of dollars of taxpayer money, major institutions, etc. are not things you want to hack and discover that whoops, you just incentivized the wrong thing and set back cancer research for a decade.

Re:Cracking down on research fraud by marcus_holmes on Monday July 27, 2020 @02:46AM (#23962746)
So, what's the best proposal or line of thinking coming from all this deep thought about how to solve it?

Because you can't possibly be saying that the current system is the best we can do, or that the problem is intractable.

(I agree with most of your proposals about software engineering btw. We should do more of this).

Re:Cracking down on research fraud by adminprof on Monday July 27, 2020 @12:49PM (#23966858)
It'd be a process: participant in a community in a subfield, listen to ideas that have been tried and outcomes, come up with an idea using those lessons learned and success metrics, build consensus around a new idea, test in a single-instance (like one conference, grant review panel, tenure committee at one university), share lessons to the field, do this for a couple of years to show clear success, expand to multiple events in the field, become an exemplar field for that idea and "infect" other fields

Sounds slow but there's thousands of such experiments happening simultaneously right now. This is how a long of major field-sized changes have happened, like the transition to conferences from journals (which had many initial problems like during tenure review or a lack of quality in reviews), etc. Ideas will lose traction at various stages (for example, there was a movement some time ago to use alpha=0.001 instead of alpha=0.05 for null hypothesis testing, which has been limited to that field or subfield).

Re:Cracking down on research fraud by marcus_holmes on Monday July 27, 2020 @02:05PM (#23967719)
Interesting, thanks.

Is the move to pre-publishing servers (like arxiv) a part of this? How does SciHub (and similar) figure into it?

It does sound slow, and a bit trivial, if I'm honest. Are there any examples you can share of successful experiments that have travelled across field boundaries?

Re:Cracking down on research fraud by mturmon on Monday July 27, 2020 @11:38AM (#23966123)
You're right. Love your example.

HN does not have much actual collective expertise in this area (say: governance structures for technical research), but HN is solution-oriented, so ideas will be proposed. They just tend to be not very good. (See also: HN climate-studies threads.)

I've spent 10 or 15 minutes on this thread, and didn't read any comments actually building on what the OP said. OP's author does have expertise in this - he has been writing a series of good stories for Science magazine in this general area. Sigh.

Re:Cracking down on research fraud by roel_v on Monday July 27, 2020 @04:47AM (#23963307)
Thank you for writing this. Your examples were lethargic. I know I should stay away from the 'academia is broken duh duh' threads here but I get sucked into them anyway. They're always full of armchair experts who don't know anything about universities or research except from being students or at best junior researchers and then think this makes them experts on the matter.

But the article itself had that same vibe for me, especially with how cock sure it is that the 'questionable research practices' are 'fraud'. I mean, come on - someone making up the responses of a 500 people questionnaire; yes that I could call 'fraud'. But not being included as an author on a paper, or being included when you didn't contribute that much? I've been in both situations and in some cases, I was completely fine with it; in others I was a bit miffed (mostly because of the same interpersonal frictions that happen everywhere where people work together) - but in none of those I would call it anywhere near 'fraud' or even 'dishonest'. Yes, there exist people who pay 1 or 2 people to write papers for them and then publish those papers with themselves as the only author. Again, that I would call 'fraud'. But the 99.9% of other cases - not even close. Just like because there is one billionaire underage sex trafficer, doesn't mean all of them are and that 'the system' is 'broken'.

And this is how I, again, got sucked into a completely non-productive 'discussion' that is so far removed from reality so as to be completely irrelevant anyway...

Re:Cracking down on research fraud by webmaven on Monday July 27, 2020 @09:13AM (#23964729)
> But not being included as an author on a paper, or being included when you didn't contribute that much? I've been in both situations and in some cases, I was completely fine with it; in others I was a bit miffed (mostly because of the same interpersonal frictions that happen everywhere where people work together) - but in none of those I would call it anywhere near 'fraud' or even 'dishonest'.

As I understand it, ghosting becomes a more significant issue when it enables the omitted author to peer-review their co-authors' papers (and vice-versa) without disclosure of the conflict of interest.

Re:Cracking down on research fraud by roel_v on Monday July 27, 2020 @09:52AM (#23965076)
shrug Sure, theoretically, but someone who wants to ensure a positive review can just as easily collude with someone who didn't do any of the work. And then still - yes it's possible that a completely bogus paper gets signed off on by a dishonest conspirator, but the editor should see the discrepancy between that and the reviews of the other authors, and dig deeper. But the real problems start in the grey zone; like when you're on the fence between 'reject' and 'major revisions'. There is no 'objective' truth there and quite honestly as an author it's a crap shoot and always has been. It sucks the first few times it happens but none of leads to 'peer review being broken'.
Re:Cracking down on research fraud by nl on Tuesday July 28, 2020 @04:03AM (#23973477)
To add to this - there are plenty of issues with peer review, but someone deliberately avoiding an authorship so they can peer review seems low down the list of real world problems.
Re:Cracking down on research fraud by Chris2048 on Monday July 27, 2020 @12:38PM (#23966734)
> If it was this simple that a random person here could come up with how to "solve" academia, we'd have already done it decades ago.

Just because an idea is simple to come up with, doesn't mean it's also easy to implement.

Also, can you explain to me how your programming analogy works? Because I agree with a lot of it, though it seems I'm not supposed to.

Re:Cracking down on research fraud by adminprof on Monday July 27, 2020 @12:56PM (#23966919)
Sure I guess I didn't make that very clear. Just going down the line:

1) stop releasing before the bugs are fixed. Software and games are rushed out. Companies need to take their time to fix the bugs so the users don't have to encounter them.

- This is a tradeoff between shipping time and bugs. You will never fix all bugs, so it's unrealistic. And it's a business decision in many cases. Even the definition of a bug is tricky, like is a usability issue a bug? So it's just a naive idea.

2) no more technical debt. Programmers are sloppy and introduce technical debt because they are not incentivized to do high quality programs.

- No one wants to create technical debt. Obviously it slows down development later on. Again this could be a business decision. Some programs like one-off data science scripts don't need to fix all their technical debt. Technical debt also accrues naturally (like just changing environment, platform, standards) so it's not possible to aim to not have debt in the very beginning. Hindsight is 20-20 and all that. Saying programmers are not incentivized to do high quality programs is just a blanket naive statement, and depends on the definition of high quality programs.

3) cap team sizes. Everyone knows that large teams fail more spectacularly. Gmail and Napster and the original version of Google were made by a group of 4 people. Software teams need to be 4-5 people max.

- Depends on the type of software. Can't just generalize given a few token examples. Expectations also change over the course of the product.

4) programmers must use a transparency scorecard. Software companies like Oracle and IBM charge ridiculous amounts for their work. They hide costs and cut corners. Programmers should be transparent about the work they are doing each day, what data they access, and which functions they are writing.

- This uses one subsection of the software economy to make a point (as a fallacy). But also some of these measures don't make sense, like some programmers read a lot of code or delete lines, and so the metrics are not generalizable.

In summary, these are ideas that someone who has not done long-term software development would say, or someone who has only had experience with one type of software or company would say. They're not well defined, not generalizable, and don't account for the complex and varied sociotechnical process that software development is.

Re:Cracking down on research fraud by gwerbret on Sunday July 26, 2020 @09:07PM (#23961421)
1) I think the idea of funding "people, not projects" (the Howard Hughes Research Institute's byline; I'm not affiliated with them in any way) is a great one. It would probably involve changing the structure of research programs in other fundamental ways, such as by

2) Limiting the number of graduate students. The current relatively-low barrier of entry into grad school provides cheap, motivated labor for PIs who are trying to stretch their research dollars to the limit. The outcome today is far more PhD graduates than jobs, and for those who get jobs, far too many of them for the research funds that are available. The overall quality of research drops when PIs focus what is fundable, rather than what is important.

3) Shifting the burden of the bulk of research work from trainees to salaried research associates/assistants/lab techs.

4) Change the focus of research output from novelty and volume (of papers published) to quality and significance of work done. Good research is often slow and careful, and doesn't fit well with the demands of grant funding agencies.

Fewer grad students means more resources available for their training, and better career prospects. Shifting the bulk of the research benchwork to salaried professionals removes the incentive to commit fraud. None of this has to come at the expense of research quality; as an example, the NIH/HHMI already have approaches to vetting research programs for quality, even if the PIs aren't competing for grant funding.

Re:Cracking down on research fraud by an_opabinia on Sunday July 26, 2020 @09:11PM (#23961436)
> Limiting the number of graduate students.

Isn't a lot of the graduate school product providing a way for people to immigrate to the United States?

The economy around it is pretty complicated and definitely there's way more pressure to supply more, not fewer, spots.

Re:Cracking down on research fraud by vkou on Sunday July 26, 2020 @09:57PM (#23961657)
No, the problem is a bit more structural then that.

A single professor trains dozens of grad students over the course of their career. Even if you remove every single foreigner, and every single person who does not want to pursue a career in academia, you still end up with dozens of graduates - per professor.

The number of jobs available for those graduates?

One - that professor's - when he or she retires. And until then, they get to burn the midnight oil, doing grunt work on their projects, in the hopes of competing their dozen collegaues for a shot at that one spot.

Re:Cracking down on research fraud by nl on Sunday July 26, 2020 @11:46PM (#23962064)
> The number of jobs available for those graduates? One - that professor's - when he or she retires.

This simply isn't true. Remaining in academia is only one of many career paths for graduate students, and many are financially compensated much better.

Some examples of these include physics and math grad students being recruited by hedge funds, comp sci grad students being recruited by tech companies and geology grad students being recruited by oil, gas and resource companies.

Even within academia the size of the market isn't static. New and emerging universities are hungry for qualified research professors and many come from these programs.

Re:Cracking down on research fraud by vkou on Sunday July 26, 2020 @11:53PM (#23962103)
I already excluded the people who are getting a degree because they want to go to industry with it.

Of the 'stay-in-academia' camp, the ratio is hideously stacked against the graduates.

New and emerging universities aren't doubling the academic jobs pool every five years... Which is what it would take to keep up with the graduates being churned out.

(And incidentally, they are also contributing to the oversupply, because their professors will also be churning out even more new graduates.)

Re:Cracking down on research fraud by voxl on Sunday July 26, 2020 @11:05PM (#23961903)
Your second and third suggestions will literally kill all research at anything but the most prestigious universities. It is elitism. Graduate students are not always RAs, they're usually not. They often teach, so you're not giving an accurate depiction of the incentives placed on them.

Becoming a graduate student is not easy, the bar is not relatively low, and making it through the program is even harder. It's true that academic positions are not large enough but you're ignoring the private sector.

Research scientists could not possibly be paid enough by universities. These would be graduate students minus the mentorship. A worse of all worlds.

Re:Cracking down on research fraud by cscurmudgeon on Sunday July 26, 2020 @08:42PM (#23961308)
Research publication, funding, and jobs should not be tied to outcomes of research.

They should be tied to how systematic, logical, and well-documented the research is. We need a system wide change from funding bodies, job committees to publishing criteria.

Re:Cracking down on research fraud by mlthoughts2018 on Sunday July 26, 2020 @08:46PM (#23961327)
Along the same lines, “what question to ask” or “what idea to try” should never generate credit or fame for individuals. This incentivizes Indiana Jones style research, results at all costs.

As a researcher you should be desiring to get jobs or rewards based solely on the care of methodology, clarity of communication and ease of reproducibility.

If you do those things well and the papers turn up negative results, that’s good archived knowledge for society. Turning up positive results should be viewed as an emergent property of a wide network of labs, agencies, universities and governments, and never a property of darling individuals.

Re:Cracking down on research fraud by dasudasu on Sunday July 26, 2020 @09:15PM (#23961457)
It can be argued that part of being a good researcher is developing an intuition and taste for interesting and promising research questions. Of course, the counterpart to this is that you see hype and fads developing very easily. But overall it is very easy to come up with experiments that any experienced researcher would have told you beforehand had no chance of producing interesting / impactful / positive results. The system must discriminates against researchers that are able to come up with interesting (positive) results with greater probability, since it represents every time an expense of finite resources. If your only contribution as a researcher is to be able to make conscientious and well realized experiments, then sadly your place is much more as an assistant than a PI.

I've never seen it argued in serious research circles that grants should be given this way, but I did find the non-research community to be overly obsessed over negative results not being published or rewarded. It is simply a consequence of the set of possible experiments being infinite, the same way you can come up with an infinite number of startup ideas, but not all of them are equally good even before implementation. Should we reward startup founders with failed startups because they tried their best and really did the best they could given the circumstances?

Re:Cracking down on research fraud by mlthoughts2018 on Sunday July 26, 2020 @11:18PM (#23961961)
The comparison with startups isn’t valid - they seek an explicit profit outcome where good or bad ideas are judged by success in markets. Research is meant to be a public good, that furthers society’s at large repository of knowledge and culture.

> “ It can be argued that part of being a good researcher is developing an intuition and taste for interesting and promising research questions.”

I don’t think this can be argued actually. This is mythology, usually applied to creditmongers who run labs and accumulate accolades that are actually due to a wide array of students and post-docs who are made to get reduced credit as a type of dues paying laced with rampant discrimination and sexism.

I’d say mythologizing the idea of a crack sleuth who has a special knack for research intuition is extremely harmful on all fronts: depriving value to society because it’s false and depriving value to the wide network of lower level staff who are actually responsible for progress.