Could it be there's more than one problem? :)

Social science is non-replicable because the parameters are too fuzzy. (People are more diverse than electrons).

Lab science is non-replicable because the experiments are too expensive to set up, and sometimes are intentionally kept as trade secrets for economic reasons.

The pressure in most of these sciences is to consistently get a paper for every expensive experiment that is done, but if you're doing that, and published work has to be a positive result, you're not really doing science. You could just publish your hypothesis and be done with it if you already know the result of the experiment will be ginned up in some way to be positive.

I don't see how we do this. Ultimately, the buck stops with the public. Governments fund the sciences because the public demands it. If scientists spend more time reporting negative or ambiguous results then the public begin to question what they're paying for.

People want breakthroughs and scientific revolutions, not "business as usual." And this bias is not limited to the public, scientists themselves want to see new science.

Is this a scientific fact? Secondly, is it an epistemically flawless fact (which is similar, but also different)?

In what way exactly does "the public" "demand" that which science does?

Why don’t you check on Google Scholar and report back to me?

In what way exactly does "the public" "demand" that which science does?

In the economic way that we say most things are demanded. I’m not saying that people are going to be out in the streets protesting if scientists start publishing more negative results. I’m saying that the general public attitude towards science (the one measured in polls) is affected by the public’s perception that scientists are producing important and meaningful results.

Standard conventions for burden of proof I suppose. Surely you don't actually have no evidence to back up your claim, do you?

> In the economic way that we say most things are demanded.

Are you discerning support for science from economic data of some sort?

>I’m saying that the general public attitude towards science (the one measured in polls) is affected by the public’s perception that scientists are producing important and meaningful results.

And might it be possible that the public's perception of science could be affected by all of the free marketing it's gotten in the last five or so years?

So I don't get how this supports the current framework.

This is basically just noting Goodhart's Law, or as I like to say "Warning." Evaluation is incredibly difficult and every metric can be hacked. Making "meritocracy" a fool's errand since a metric alone is not enough (it needs the associated uncertainty value, which is non-zero).

The problem here is that we're searching through an incredibly high dimensional non-convex solution space and assuming that a simple optimization function is a global optimizer.

Instead I'd just love to see academics pursuing their ideas and discussing them in the open. I don't think you'd have a significant problem getting people to generally align in a single direction, as currently happens, but I think it also allows for others to challenge those directions as well. Essentially research is performing some sampling in that search space but we're turning the learning rate way down. So we get plenty of works where novelty and incrementalism is indeterminable, due to the speed in which we must produce. Instead let the academics determine their own parameters and let time sort it out. Ensembling is a very useful technique, especially when uncertainty is high.

This just advocates for a system that more accurately captures uncertainty in the models. Which, would technically make the results replicable but reduce their impact. At least in the short term. Impact in the long term is harder to determine.

Though, I really think A LOT of fields need to be far more honest about their levels of uncertainty and any ones that aren't (especially empirically based ones) are bordering snake oil territory.

Social scientists tolerate their peers who see themselves as agents of change more than as scientists. A Stanford education specialist, who wrote California's new mathematics standards for public schools, framed--among other things--a well-reasoned takedown of one of her papers [1] as personal attacks [2].

Consider the same happening in physics. Paper published. Critique follows. Is there any world where a physicist maintains respectability by ignoring the content of the critique while claiming harassment?

Yet that's not only tolerated in many social sciences, it's the norm. Add to that lucrative consulting gigs, and researchers have a perverse incentive to (a) extrapolate N=2 studies of hung-over undergrads to corporate America and geopolitics and (b) personally attack anyone who dares challenge their premise.

[1] https://www.nonpartisaneducation.org/Review/Articles/v8n1.pd...

[2] https://joboaler.people.stanford.edu/sites/g/files/sbiybj286...

This isn't true at all. It's quite easy to create well-defined survey questions and experiments. It's entirely replicable.

The main problems with social sciences are that they have historically been perfectly fine publishing studies based on statistically tiny numbers of participants, as well as falsely extrapolating findings from tiny groups such as university undergrad test subjects to all of humanity.

These problems are eminently solvable with stricter statistical standards, and much greater cross-cultural research. It's just that these studies are massively more expensive to run.

I don't know that anyone else does, but I like to draw a distinction between experimental scientific inquiry and rigorously observed phenomenology. Properly speaking I say a great deal of what's called science is really a species of phenomenology. For example cosmology is pure phenomenology. We can't actually run experiments on what happens when you smash two black holes together. However we can rigorously observe the phenomenon and abduce a descriptive model that can potentially even make predictions. But the word "science" has become a totem, so there's a lot of cargo culting[1] and punishing blasphemers.

However, I don't really think that whether a field is science, phenomenology, or some blend of both has much effect on the rate of fraud. Rather, I hypothesize the dominant factor in the incidence of fraud is the perceived magnitude of the potential benefit to be obtained. The Alzheimer's fraud is a good example of that. There are hundreds of billions and maybe even trillions of dollars in play in that space. Drug companies have already poured billions down the drain chasing dead ends so anyone who can sell them hope can potentially get a substantial payday. An amusing corollary of this is that given the relatively small stakes social science fraudsters are playing for, we can conclude that they are small-minded in the sense that they aren't thinking nearly as big as the drug company fraudsters.

[1] https://calteches.library.caltech.edu/51/2/CargoCult.htm

https://en.wikipedia.org/wiki/Grievance_studies_affair