Point well taken. The a-theoretical nature of the social sciences (and many others) has come up before. "A problem in theory" discusses this:
As for science, my main point is that we need to embrace the noise. A lot of people try to use metrics to reduce the noise (which is good) but have this fundamental belief that noise can be absolutely removed. But this is a deep misunderstanding of what noise is. Ignoring the probabilistic nature of the universe, even a fully deterministic system has noise. A good example is the lack of absolute precision. Statistics captures error and helps us analyze that error. But error always exists due to the lack of both omniscience and omnipotence. In an nondeterministic world, it just becomes more important to embrace it. To acknowledge that it is a fundamental part of not just the model, but the thing we're modeling. In this respect, removing a noise variable doesn't just result in overconfident results, but missing a fundamental variable. Of course, noise can also be used to capture unknowns. So in any case, the noise has the be embraced. We should reduce it as much as possible, but it can't ever be removed. That's the problem: people are removing noise from their models (e.g. using a benchmark to determine evaluation absolutely)
As to your other comments and blog, I'm well aware. And if I may add some constructive criticism, if you want to call something pixelwise identical, it helps to show a differencing map. It just makes the claim substantially stronger and clearer, especially since there's a lot of additional factors that can change pixels, making the likelihood of a match even lower. These are things that wouldn't be possible by their explanation, whereas human interpretation is more subjective (it'll be a bit messier than this though). Though even low differences are still pretty damning.
> with the known nature of how to scientifically answer questions.
I think this is a bit naive though. Generally in science we rule things out, not rule in. That's why I'd be hard pressed to say that we have a "known nature of how to scientifically answer questions" because "scientifically" is doing a lot of heavy lifting and is fuzzy. Essentially "there's wrong ways to do science, but not a clear 'right way.'" Obviously faking data isn't okay nor is just doing something and calling it an experiment. Rigor is necessary but what constitutes sufficient rigor is dependent on the strength of the claims and thus the uncertainty of those claims. A big issue we face is that we take empirical results as equivalent to analytically derived theoretical models combined with experimental results. i.e. Physics develops a theoretical mathematical model and tests experiments against that whereas {psych,medicine,ML} make a non-mathematical hypothesis and report results. The problem with the latter is that it is harder to capture uncertainty because it isn't as strictly bounded. But it is still science, and the reason this is done is because the models are likely intractable (if not provably so). But that's why Asimov wrote about the Relativity of Wrong, rather than the precise way in which we can exactly do science. In fact, if the latter was true, we'd program machines to do it. Just a bit more nuanced.