News for hackers. Stuff that matters.
Re:Undisclosed tinkering in Excel behind economics paper by thimp on Sunday February 11, 2024 @05:01AM (#39333901)
Oh this is nothing. Go look at the social sciences. They get data that compromises their entire profession, tell the university it’s inconclusive and wait until the research grant runs out and fail to publish. Then do the same thing again.

I have seen this with my own eyes. A bit of excel drama is usually behind their published papers. Also seen that with my own eyes.

The field has trouble collecting data too because it’s expensive and very subjective. The worst one I saw was somehow turning 7 data points via imputation and interpolation into a hundred or so. The principal researcher on that I actually had to teach BIDMAS to and her calculations were completely whacked out (excel again)

Re:Undisclosed tinkering in Excel behind economics paper by screenoridesaga on Sunday February 11, 2024 @08:43AM (#39334931)
A lot of people are upset when their grant doesn’t get funded and it’s easy to suppose there’s some sort of sinister motive behind it. In truth it’s probably that the science is very difficult and only a tiny fraction of people get anywhere in social sciences.

Try marine biology, they are always looking for decent papers.

Re:Undisclosed tinkering in Excel behind economics paper by concordDance on Sunday February 11, 2024 @11:37AM (#39336145)
What profession have you seen compromised?
Re:Undisclosed tinkering in Excel behind economics paper by FrustratedMonky on Sunday February 11, 2024 @12:02PM (#39336357)
Because there are bad apples, doesn't mean you should throw out the scientific method.

Over 50% of social science studies do replicate.

Sorry that isn't high enough. We should definitely just give up on understanding ourselves.

Or, maybe fix the financial incentives.

Any industry across all fields, if the incentives are miss-matched, will produce crap.

Re:Undisclosed tinkering in Excel behind economics paper by ace2358 on Sunday February 11, 2024 @01:38PM (#39337261)
> Over 50% of social science studies do replicate

There is some tasteful irony here. Could you link to a study on this claim? And ideally a few other studies that support its claim? Thank you.

[dead]
Re:Undisclosed tinkering in Excel behind economics paper by FrustratedMonky on Sunday February 11, 2024 @02:57PM (#39338065)
Why cite something if any source will be rejected.

Where are we when no studies are trusted, and no review body of any study is trusted.

Don't think it is just social science. That is hubris, in our post truth world.

""A study published in 2018 in Nature Human Behaviour replicated 21 social and behavioral science papers from Nature and Science, finding that only about 62% could successfully reproduce original results."

Roger A (27 August 2018). "The Science Behind Social Science Gets Shaken Up—Again". Wired. Retrieved 2018-08-28. Camerer CF, Dreber A, Holzmeister F, Ho TH, Huber J, Johannesson M, et al. (September 2018). "Evaluating the replicability of social science experiments in

Nature and Science between 2010 and 2015". Nature Human Behaviour. 2 (9): 637–644. doi:10.1038/s41562-018-0399-z. PMID 31346273. S2CID 52098703.

https://newsroom.haas.berkeley.edu/research/amid-a-replicati...

https://royalsocietypublishing.org/doi/10.1098/rsos.200566

EDIT

"For instance, a major reproducibility project that sought to replicate 193 experiments from 53 high-profile cancer biology research papers ran into multiple barriers."

https://bigthink.com/hard-science/the-replication-crisis-is-...

EDIT 2:

Physics Discussion https://pubs.aip.org/physicstoday/article/72/12/8/811763/The...

Re:Undisclosed tinkering in Excel behind economics paper by thimp on Sunday February 11, 2024 @03:10PM (#39338183)
Respect for citations. I'll read into this over the next few days.

I tend to treat papers as disinteresting until someone has asked me to tear a new hole in one (I can only really do that mathematically) or they have been reproduced. Decreases cognitive load when someone says "check this out".

Re:Undisclosed tinkering in Excel behind economics paper by FrustratedMonky on Sunday February 11, 2024 @03:18PM (#39338249)
Well, that is the problem.

Time and "Decreases cognitive load".

We really trust scientific papers to have done the leg work. If it is in a respected peer reviewed publication, then we can trust it and then build on it.

We can't all individually reproduce every study.

I really fear for the future.

Because already there are tons of garbage studies being faked by AI flooding the world. It will become very difficult to know anything.

My only beef here is that it is not isolated to social sciences. HN really piles on social sciences while having a blind spot for STEM. Right when neuroscience and AI are converging and both sides need to have more understanding.

Edit:

at Minute 5

https://www.youtube.com/watch?v=g7TghURVC6Y

Re:Undisclosed tinkering in Excel behind economics paper by a_bonobo on Sunday February 11, 2024 @08:44PM (#39340568)
The poster might have confused social sciences with psychology?

>Study replication rates were 23% for the Journal of Personality and Social Psychology, 48% for Journal of Experimental Psychology: Learning, Memory, and Cognition, and 38% for Psychological Science. Studies in the field of cognitive psychology had a higher replication rate (50%) than studies in the field of social psychology (25%).

https://en.wikipedia.org/wiki/Replication_crisis

Definitely better than medicine!

>Of 49 medical studies from 1990 to 2003 with more than 1000 citations, 92% found that the studied therapies were effective. Of these studies, 16% were contradicted by subsequent studies, 16% had found stronger effects than did subsequent studies, 44% were replicated, and 24% remained largely unchallenged.[76] A 2011 analysis by researchers with pharmaceutical company Bayer found that, at most, a quarter of Bayer's in-house findings replicated the original results.[77] But the analysis of Bayer's results found that the results that did replicate could often be successfully used for clinical applications.[78]

25% of experiments replicating in-house is crazy low, but explainable due to very different incentives (the researchers need to get the paper out; the Bayer scientists need to have a functioning product)

Re:Undisclosed tinkering in Excel behind economics paper by FrustratedMonky on Monday February 12, 2024 @07:11AM (#39343954)
Didn't realize there was so much variation from field to field. Even inside 'social sciences'.

I'd like to see this across all, including STEM.

Note: I'm using 'social science' colloquially. On HN, seems like Psychology is a 'social science'. Anything dealing with people is deemed a 'social science'. Even though that isn't technically correct, they are separate fields. There is a 'Social Science' field and a 'Psychology' field.