https://slatestarcodex.com/2014/04/28/the-control-group-is-o...
On the far extreme, you have stuff like the Hubble telescope[1], LHC[2], or LIGO[3], all of which faced and overcame extreme technical challenges. Do you know how much work it took to get LIGO isolated from outside vibrations? Literally a decade.
[1]: https://www.nasa.gov/content/hubbles-mirror-flaw
[2]: https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2014...
[3]: https://www.ligo.caltech.edu/page/vibration-isolation
Or consider this XKCD comic[4] about geology. "We study what's on the other side of a 20 mile thick rock wall." Similar problems exist for cosmologists and astronomers, but somehow scientists are taking pictures of black holes[5] and finding exoplanets[6].
[5]: https://www.jpl.nasa.gov/edu/news/2019/4/19/how-scientists-c...
[6]: https://en.wikipedia.org/wiki/Exoplanet
What psychology experiments face the same kind of obstacles as these? Your subjects are not billions of light years away, nor smaller than an electron, nor whizzing by at just under the speed of light, or behind 20 miles of rock.
Yet the current state of psychology research[7] is that maybe 1/3 or 1/4 of papers can be replicated. When researchers start to explore why, they find that QRPs (questionable research practices) are common, sample sizes are small, and statistical analyses are often misused, and so on. In other words, they find the methodology is at fault, not the fundamental difficulty of the subject matter.
I'm not a scientist or researcher but can't this all be explained by the fact that we know much, much more about these fields than we do the human brain? And therefore, isn't it likely that at least some of the difficulty in reproducing results can be attributed to as-of-yet unknown mechanisms of the brain?
> In other words, they find the methodology is at fault, not the fundamental difficulty of the subject matter.
Why not both? And who precisely is claiming it's not a difficult subject matter besides yourself?
It is a messy area of work. Brains work differently between people, they work differently over time, among different groups of people, and they can rarely (outside of functional MRIs where you have structural factors) have anything but self reported outcomes. You have the privilege of saying that because people post their research and their numbers, which is all you can ask from any field of science.