News for hackers. Stuff that matters.
Re:50% of neuroscience papers suffer from a major statistica... by scott_s on Monday November 28, 2011 @02:58PM (#3287341)
1. Outright forgery or manipulation of data is a huge black mark on a researcher. Huge. It lasts forever, and is much bigger than the prestige from a single published article, where ever it was published.

2. http://en.wikipedia.org/wiki/Hanlons_razor

Re:50% of neuroscience papers suffer from a major statistica... by forensic on Monday November 28, 2011 @04:34PM (#3287732)
Every scientist who has been caught forging data, has basically just claimed it was an accident. Whether they got "punished" or not depended purely on their political connections, not their behaviour.

What's worse than the possibility of a black mark on your career? Not having a career at all. Which is what will happen if you don't publish.

Re:50% of neuroscience papers suffer from a major statistica... by scott_s on Monday November 28, 2011 @04:46PM (#3287765)
You're basically making an economic argument, where you assume the risks for cheating outweigh the consequences of being caught. But your support for that argument is the claim that the risk of being caught forging data is less than the risk of not having a career, but that is a false dichotomy. There is a middle ground, which is publishing valid but less sensational results.

In other words, the choice is never "Forge these results, or have no career."

Re:50% of neuroscience papers suffer from a major statistica... by forensic on Monday November 28, 2011 @07:27PM (#3288360)
> In other words, the choice is never "Forge these results, or have no career."

Yes it is, because in order to have a career, it is not enough to just publish. You have to publish in prestigious journals and you have to publish original, groundbreaking, important, and meaningful research.

A tiny percentage of academics actually end up keeping their job until retirement. If you don't plan on perishing, you publish in prestigious journals by whatever means necessary. Whatever. Means. Necessary.

"Accidentally" using bad statistics is the kind of "accident" that gets you tenure. It's significantly easier to forge groundbreaking, important, prestigious research than it is to actually do this research.

Scientific fraud is epidemic precisely because of the economics of universities.

Re:50% of neuroscience papers suffer from a major statistica... by kd1220 on Monday November 28, 2011 @08:27PM (#3288517)
I experienced this mindset in my graduate studies. Many graduate advisors spend so much time writing grant proposals and hobnobbing with government project managers that they didn't care what result their experiments got, as long as it was positive. I lost interest in being a researcher primarily because it's no different than working at a company - except you get paid less.

I never saw any researchers or students falsify or make up data, but often I saw experiments performed without defining a hypothesis first. The hypothesis was created a posteriori to match any interesting correlations that could be combed from the data. I felt like I was a marketer trying to find a reason to promote a product. This particular approach might be standard when dealing with in-vivo experiments (on human subjects). It's a time-intensive process and you can't just rerun the experiment like you can on an artificial system. However, I didn't feel like it was the correct way to do research.

There's been renewed interest in recent years of creating a journal of failed experiments for various research areas, but it never seems to catch on. It would be invaluable, but there are too many egotistical people in academia who don't want to be associated with failure in any way.

Re:50% of neuroscience papers suffer from a major statistica... by PakG1 on Tuesday November 29, 2011 @05:18AM (#3289794)
Re:50% of neuroscience papers suffer from a major statistica... by scott_s on Monday November 28, 2011 @09:06PM (#3288630)
I know many, many researchers, and very, very few who have published anything "groundbreaking." Meaningful, original, often even important, but rarely groundbreaking. There are different tiers of venues to publish in, just as there are different tiers of institutions to do research in.

I think the economics often are perverse, but I think you're doing a disservice to the overall discussion by simplifying it too much.

Re:50% of neuroscience papers suffer from a major statistica... by forensic on Tuesday November 29, 2011 @04:38AM (#3289731)
You're splitting hairs. Groundbreaking has a pretty low bar these days.

Just remember that 50 year old profs came of age in a different era.

Re:50% of neuroscience papers suffer from a major statistica... by scott_s on Tuesday November 29, 2011 @04:14PM (#3292199)
Huh? As a field matures, the bar for groundbreaking increases. Most research is iterative, including everything I've done, and almost everything my peers have done.
Re:50% of neuroscience papers suffer from a major statistica... by forensic on Tuesday November 29, 2011 @11:33PM (#3293737)
Why are you stating the obvious?

I am trying to make a point that in the world of vicious competition for grant money, every shitty incremental experiment is lauded as groundbreaking. For very understandable reasons.

I'm being facetious about the way that the key quality of a scientist has shifted from doing and teaching science to being a great marketer who can win grants. The "best" scientists these days are generally the ones who are really great at hype.

Re:50% of neuroscience papers suffer from a major statistica... by scott_s on Thursday December 01, 2011 @03:59PM (#3301365)
It was not clear that you were using "groundbreaking" facetiously. Hence my confusion.

I agree there are problems with how science works right now, and I agree on what the problems are. I disagree on the weights of the incentives, though. You describe "rampant" fraud. I have observed none. Then my conclusion is that the claim that fraud is rampant is incorrect.

Re:50% of neuroscience papers suffer from a major statistica... by forensic on Saturday December 03, 2011 @11:09PM (#3309841)
>I have observed none.

How hard have you looked and what is your definition of fraud?

Re:50% of neuroscience papers suffer from a major statistica... by joe_the_user on Monday November 28, 2011 @07:30PM (#3288366)
Having noticed "Hanlon's Razor" becoming the default reaction for a number of HN posters, I have to say what I feel is really bad about it when it applied to complex human endeavors.

When one deals with modern institutional corruption, what one is dealing with is a complex combination of incompetence, overt deception of others, self-deception and simulated incompetence.

Sure, incompetence is in the mix but when you use a construct like "Hanlon's Razor" to "label and forget" the situation, you hide the full situation. The complexities of these situations are there, there is no razor for a priori untangling.

Re:50% of neuroscience papers suffer from a major statistica... by scott_s on Monday November 28, 2011 @09:02PM (#3288623)
I think of it as a bias towards suspecting incompetence rather than a bias towards suspecting malice. I think people are too quick to assign narrative to situations, and malice is a stronger narrative than incompetence.