News for hackers. Stuff that matters.
Re:Marc Andreessen on Investing and Tech by nonameiguess on Wednesday August 18, 2021 @12:57PM (#28223985)
Not only this, but it appears to me that diagnoses of the state of science as a problem are making the classic problem a person like Andrew Gelman would usually quickly point out, which is ignoring base rates. Saying 70% of published research can't replicate sounds bad in a vacuum, but how does that compare to 50 years ago? We have no idea if this number that sounds terrible outside of any context represents an improvement, degradation, or no difference at all from what we should expect.

That isn't by any means to suggest we shouldn't try to make science better by being more replicable and reducing fraud, but if it is an improvement, we're already doing something right and there is far less cause for despair and cynicism.

It also, of course, doesn't do any base rate comparison to non-scientific ways of generating facts. If 90% of published psychology research turns out to be wrong, but 99% of folk psychology turns out to be wrong, then science is still winning there.

Re:Marc Andreessen on Investing and Tech by jollybean on Wednesday August 18, 2021 @01:35PM (#28224565)
"Saying 70% of published research can't replicate sounds bad in a vacuum, but how does that compare to 50 years ago?"

It doesn't matter a whole lot with respect to 'how it compares to 50 years ago' if the assumption that science is supposed to be objective and credible.

It's supposed to be 'approaching 100%' i.e. they are publishing findings, not fake findings.

i.e. it's not a game of 'getting a higher percentage' it's a game of integrity.

The implications that 'most science is rubbish' is pretty scary actually and it has profound consequences for the future both in practical terms (i.e. we can't trust a lot of it), but especially in populist terms (i.e. people not trusting climate change, vaccine research etc.).

"If 90% of published psychology research turns out to be wrong, but 99% of folk psychology turns out to be wrong, then science is still winning there. "

Is it though? We spend bazillions on psychology and nothing on pop psychology, so we'd have to contemplate how valuable that '10%' is to society - especially in the context that we may not even know 'which 10%'.

This is kind of a big problem.

Re:Marc Andreessen on Investing and Tech by majormajor on Wednesday August 18, 2021 @01:56PM (#28224790)
There's a big, important difference between "fake fraudulent findings" and "not-replicable findings." The latter suggests problems stemming from a big nasty complicated poorly-understood world, and can be improved with introspection and change over time. The first step is the greatly increased discussion over it now than in the past - we're discovering the size of the problem. The next step will be to do things with that. Publishing negative results, getting everyone to have a better understanding of prob/stat... I think those are the directions to go in.

If, on the other hand, you just dumped all those psychology researchers out of the academic environment and sent them to marketing and advertising companies... are we better off?

Re:Marc Andreessen on Investing and Tech by TapWaterBandit on Wednesday August 18, 2021 @08:04PM (#28228785)
> If, on the other hand, you just dumped all those psychology researchers out of the academic environment and sent them to marketing and advertising companies... are we better off?

Honestly? Very possibly.

The reason is that regardless of your opinion of marketing/advertising (mine is not overly positive while acknowledging that informing consumers of new products etc is a worthwhile endeavour broadly speaking) at least if the psychology researchers were trying to implement whatever their pet theory is in a company the market will quite quickly let them know whether or not it is horseshit. Markets have faults of course but at least if the psychological theories were implemented at companies you could reasonably easily tell if they were effective or not simply based on do they increase sales etc. If they turn out to be nonsense, this acts as a signal to the individual as well as others that this avenue of study is not worth further exploration.

Whereas Academia can often act as an artificial greenhouse environment for theories/hypotheses that won't survive contact with the real world but seem quite successful in the artificial environment.