Shipping truth that isn’t powerful isn’t as attractive to your customers as truth that is. So you’re incentivized to develop truth that is. And you do this by hacking the accepted standards in knowledge pursuit by starting with a conclusion and working backwards. You tell a story based on the evidence that’s useful.
Now it’s “the science” or at the very least “there’s studies” and this is useful to both the customers (NGOs, journals, activists, lobbyists, media, anyone that wants to influence policy) and the university (attracts money, reputation and status) and the people shipping it (tenure, book deals, speaking fees).
It’s not a conspiracy. It’s just simple incentives. The poor guy who spends his time figuring out what the truth is not or that ships truth that isn’t immediately useful to the customers is looking for a new job after his grant dries up.
So it's not even about what is or is not true, or what is recognized as useful in the short term, it's that what defines truth at any given time is what is popularly believed to be true.
This sounds very flaky but it has real consequences because the incentives aren't just to find positive results, it's to find positive results that are likely to be accepted by the majority of the academic field. This kind of positive feedback loop is bound to lead to pressure because science almost by definition has to get some things incorrect — it's how progress is made. So people are actively punished for going against the grain, or exploring areas where our knowledge is limited and therefore prone to lots of negative results because we just don't know.
That’s the job of the prestigious media. They manufacture consent to the masses. They propagate right-think. Again, not a conspiracy but simply incentives.
Anyways, there is objective truth. Not everything can be settled that way though and you’re right in that “truth” can mean consensus in many instances.
The problem with that is that in many things the dissent is tasked with proving claims untrue, because of the consensus, rather than those making claim to knowledge having to prove it is true. There’s a lot of this today.
For humanities and social sciences, it could mean consensus, but in hard science it is different. You are doing experiments that produce data. If someone else repeats the experiment, they should get the same data.
In this case, no one could reproduce the results of the papers in question.
I wonder how widespread this has become. How many scientists, driven to publish, know that it is too expensive to reproduce their experiment?
Incentive structures make sure that the latter is dominant in published science.
Even if there was no intentional fudging of data, results and interpretations by scientist teams, the results of science as a whole could still be heavily (over time) influenced by simple biases in selection of grant applications. For example if you only fund researches of type "thing X good" and dismiss all applications trying to prove "thing X bad", you will eventually gather enough evidence for X to appear good, regardless of how objectively "good" it is.
Not just a contest of hiding doping, but of winning against others who were doping. If everyone is doping, then you could argue it's still a fair playing field, just with a higher skill ceiling.
No, because now you have at least two contests in parallel: the race event for the cyclist and through-the-year "doping and not getting caught" for medical/management team.
For example one prominent olympic level swimmer was disqualified because they failed to appear for testing multiple times. Why would one do that just to come back after disqualification?
Churches, Governments, Academia. Hierarchies form, incentives drift over time and become mismatched with original intent. The proof is in history, look at Church abuse, Inquisition, look at grade fixing, admissions scandals, look at Volkswagen tweaking emissions, look at CEO's lying to promote products. Our entire society is based on the ebb and flow of some kind of corruption, or ebb-flow of Law.
Even today, in America, half the population think that the laws themselves are corrupt and should be broken. Judges aren't trusted. So who is corrupt if you don't have any foundation to judge anything against?
I'm totally against China taking Taiwan, but from a 'certain point of view', they have point. It was a part of China before the war, and the mainland 'won'. So they think of it as unification.
Again. I am totally against China taking Taiwan. Just think it is bit more nuanced.
Actually, think we are agreeing.
So, we should call out all cases. Even if there are so many cases to cause one to become tired of the fight.
Maybe I was just reacting that in todays world the lines have blurred enough that it is sometimes hard to tell if you are fighting corruption, or if you have become the corruption.
From different points of view both these are correct:
"The US experienced a Coup attempt by a president that then did not suffer any consequences for his un-constitutional actions"
or
"The President, fighting for the American Way against the corrupt deep state government attempted to bypass a fake election to continue his quest to root out evil? "
This is not true. Even during the 1904 Tour de France where 9 people were disqualifed because of, among other actions, illegal use of cars or trains [1] - 27 riders finished the race.
Tour de France in the modern era has up to 180+ competitors lining up, and there hasn't been a case of 100+ riders being disqualified for doping.
[1] https://en.wikipedia.org/wiki/1904_Tour_de_France#Disqualifi...
There, the claim is that of the top 10 finishers, all but one participant were either stripped of result in that race or failed tests/were sanctioned for doping in another race.
(The remaining participant, originally #8 Cadel Evans, was a documented client of Michele Ferrari.)
I can imagine some kind of first order phase transition in which the amount of fraudsters becomes so large that self correction breaks and instead they start to endorse each other. But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet.
In smaller sub (sub)fields sometimes fraud becomes large enough to self-sustain, like it happened with Majoranas [2,3] but even that kind of self-corrected. [1] https://arxiv.org/abs/2204.08284 [2] https://twitter.com/spinespresso/status/1503352928656138241 [3] https://espressospin.org/2022/11/17/the-fallen-angel-particl...
This one is so subtle, it's probable that many of the people doing it don't even realize they're doing something wrong.
Fake is a spectrum. Soft faking is easy and widespread--"p-hacking" is the polite term.
The stakes for faking it are low because most research is low impact, most research is seldom reproduced, and even bad research is rarely retracted. For that which is reproduced there is plausible deniability--"researcher degrees of freedom". Getting a high impact study published is more important than having one retracted, so researchers follow that incentive structure.
> But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet.
Have you taken a look at the biosciences, social sciences, and medicine? Basic research hasn't been reliable for decades.
That's contradictory. For low impact studies you can fake whatever you want, because nobody really cares. That probably wouldn't promote your career either tho. Publishing bunch of organic semiconductor papers in Nature and Science will promote your career, but won't hold. That's the point.
> Have you taken a look at the biosciences, social sciences, and medicine? Basic research hasn't been reliable for decades.
Not deeply, but I surely can come up with bio/med research with real life applications that appeared in last 2-3 decades. Say mRNA vaccines?
What is contradictory in a claim that most research is low impact thus easier to fake while the incentives for publishing high impact research skews to publishing bad/fake papers since publishing is far more important then having a paper retracted?
> Not deeply, but I surely can come up with bio/med research with real life applications that appeared in last 2-3 decades. Say mRNA vaccines?
Nobody said that all research is faked.
I have had about three people working in the research fields that have used the term, off the record of course, "Publish or perish".
Get it out the door, try to get the funding and hopefully if it works out, solve the issues before it becomes a problem.
For starters there’s so many performance enhancing substances, and entire underground industries for developing and producing them. All of these drugs would be banned by the blanket prohibition on unapproved substances, but in practice many of them don’t have any screening procedures, so won’t turn up in any tests.
There’s also the fact that many performance enhancing substances are biologically identical to substances your body is naturally producing (like bio-identical testosterone for example). The only way to test for these is to test the ratios of certain metabolites (with a margin for error to avoid false positives). In practice this basically means any athlete can do steroid cycles, as long as they are only boosting their test levels up to that of a genetically elite outlier.
You also have the therapeutic use exemption, which allows for legitimate therapeutic use of banned substances. Testosterone replacement therapy is a legitimate therapeutic use of steroids, and some sports are known for abusing TUEs for this usecase.
Testing for PEDs is way less precise than a lot of people think it is, and any motivated athlete can get away with PED use to some extent, no matter how much testing you subject them to. There’s good reason to be suspicious of a lot of athletes. One massively suspect category of athlete is all of the ones in/near their 40s who somehow manage to maintain physiques as good as/better than they had in their early 20s.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2657495/
Paper on challenges of detecting growth hormone
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3830297/
Testing for both of those substances has improved, but still leave plenty of room for abuse.
Use of "unknown" or the constant stream newly developed substances is naturally not rigorously studied, but here's a couple of articles referencing the challenges they pose.
https://thetruthaboutforensicscience.com/undetectable-perfor...
https://bleacherreport.com/articles/1955231-undetectable-the...
UFC is notorious for testosterone TUEs, but that's mostly because they regulate things differently to most other sports
https://www.espn.com/espn/otl/story/_/id/10500652/therapeuti...
Top flight soccer is another sport that's constantly garnering attention for TUEs, but most of the coverage of that is speculative, because other than high level statistics the TUE process is naturally not very transparent (and most of the recent coverage on the topic has been replaced by controversy relating to trans athletes).
Once you get far enough in ANY sport, you start considering tradeoffs between risking long term health and improving performance. If you are a competent adult, you can evaluate your drivers, consider risks.
But without fail, in ANY competitive sport, you realise that you have to risk health in order to achieve that next performance tier, there is no other way.
Once you get into that 1-10%(depending on sport) elite level, you realise that there is no way you can compete without PEDs.
I've participated in (taken myself) and have been a witness to this in weightlifting(olympic and strongman), soccer, cycling and swimming. Starting young. Real young.
Coaches of different sanity levels are trusted to make the right choices and not often questioned by busy or absent parents.
Everyone cheers on when medals are won and records are broken, without consideration for how it happens.
I personally had no idea I was taking hormone modifiers until about 16, it started at 11.
My coach just gave me what he said were vitamins and I never questioned it, he was the closest thing to a father I had, his authority was absolute.
At 16 I found myself in a hospital yellow as a banana on the brink of liver failure, that's when I found out.
Everyone on our team took what the coaches gave. Once a while we'd see a parent get into a shouting match with one of the coaches about something we didn't understand and that kid would never come back again.
Some of the guys I trained with went on to become highly recognisable names in their respective sports. One even has a supplement brand. Those I've kept in touch with continue PED use, that is the price of being at the top of the game.
My primary reason for believing this is that as a scientist myself, I've never done it, and none of my coauthors has ever proposed to do something like that. I have seen other kinds of fraud (e.g. ghost authorship) but not result manipulation.
But of course, If I were a fraudster myself I would also say that I've never done it :)
So a more general argument: scientific "doping" is just not as straightforward as cycling doping. In cycling, it's clear that if you don't get caught (and have no scruples) you benefit from doping, it makes you stronger.
In science, you first need an idea that looks original/promising, and then you do the experiments and see if the idea works or not. This kind of "doping" will only help if you (1) are good at generating ideas, but (2) these ideas often don't work. At least in my field, this is possible but far from the most common situation to be. And in other situations, there is not much benefit to faking results.
I have been joking with my former boss (who is a professor) about how easy it is to come up with good ideas. We each have five to six One Billion Dollar ideas before breakfast, and another 1-2 while in the shower! /s
Coming up with ideas is _ridicolously_ easy, executing on them is the hard part. If you're a seasoned researcher, it shouldn't be hard to guess at the "meta" of your field, you're probably aware of promising ideas discussed among your colleagues. What if you faked results, proving one of these ideas?
And the bar for cheating is _suprisingly_ low. I'm not a researcher myself, but I've experienced how my work is torn apart in peer review. But if you're dealing with data, apparently no one makes sure to actually check it before approval.[0][1]
[0] https://youtu.be/d2Tm3Yx4HWI?t=846 (analysis on the original data from the Francesca Gino scandal, which already looks suspicious without statistical analysis)
[1] https://youtu.be/KsSuhP60qnI (faked data from the Hendrik Schoen scandal at Bell Labs, as long as you make outrageous claims no one can verify, you can continue to publish in top journals)
I can see how in other fields, e.g. biomedical, execution could be more challenging and ideas more plentiful.
Anyway, I don't think ideas (read: good ones, because obviously anyone can have nonsense ideas) can be a commodity in any field. It is quite widely accepted that, e.g., when applying for an ERC grant (the highest-regarded individual research grants in Europe), having a good idea is as important as the CV if not more. This wouldn't be the case if coming up with good ideas were so trivial.
I don't know about the acceptance criteria for ERC grants, but from the peer reviews I've read from one of my professors papers it feels like there's a bias towards accepting novelty of all else. My joke with ideas was meant more general, i.e. that you can have a few billion dollar ideas, but executing on them is much harder (due to lack of funds, time or skill).
Additionally, depending on the demand of the field I can imagine it being easier to obtain funding for resarch. Imagine being in machine learning right now lol.
I wasn't talking about grant acceptance though, but purely what kind of ideas can get you published and what data you have to fake in order to get there. In the case of Francesca Gino it doesn't sound like her fraudulent resarch was so costly that she would have to have applied for large grants in order to conduct it.
> and then you do the experiments and see if the idea works or not
The problem is "if the idea works or not" can be very fuzzy. With a little p-hacking, anything can work! If nothing in your original data/experiment supports your hypothesis, you can just cherry pick some part of it that gives "significant" results.
Are you sure? Also a) how far along are you and b) how far from actual research work are you at this point?
And yes, I have more than 100 papers and in their making I haven't fabricated data or seen anyone do so. If there is fake data in any of them, it would be because someone included it behind my back. But I doubt it's the case, as during the course of my career we tried various things that didn't work and we just moved on to other experiments. I've never had a student or collaborator even suggest "let's just edit the data and publish it". The most that we have done is to try to cast negative results in a positive light to make them publishable ("the results are promising...", "the results are mixed but show improvements for some dataset..."), but without touching the data or saying anything factually incorrect.
There was at least one year's Tour de France in which all but one participant were later found to have been doping. In other words, you didn't get to that level of that competitive activity, if you didn't cheat, because it wasn't possible to outperform people with such a significant advantage. It actually became a contest, not just of bicycling, but of doping, because getting away with it was quite difficult and it took years for the world of cycling to get good enough countermeasures to shift the advantage to non-doping.
Academic research has, for decades now, been a very competitive, high stakes endeavor. Many fields have more people trying to work in it, than there are spots (i.e. grants and endowed chairs) available. If you have twice as many aspirants as slots available, and "only" 25% of them cheat (in this case, fudge their data to get more interesting results), then you get something like 50% of your field filled with fraud.
Moreover, the closer they are to the top, the higher the likelihood that they are "doping".
I live in Austin, TX, and I remember when nearly all of Lance Armstrong's competitors had been busted, but he had not yet. I said to other people, "well, that's it then, he must have been doping. You can't win the Tour de France 7 years in a row against doping opponents, if you're clean. Either doping doesn't work, or he was doping."
I recall several people disagreeing, convinced that he was clean. He wasn't clean.