Terrible analogy. It might take months to come up with an idea but another should be able to follow your method and implement it much more quickly than it took you to come up with the concept and implement it.

horrible take. Taking the LK99 situation as an example: simply copying and adapting a well described growth recipee to your own setup and lab conditions may take weeks. And how would you address situations where measurement setups only exist once on the earth? How would you do peer replication of LHC measurements? Wait for 50 years till the next super-collider is built and someone else can finally verify the results? On a smaller scale: If you need measurements at a synchrotron radiation source to replicate a measurement, is someone supposed to give up his precious measurement time to replicate a paper he isn't interested in? And is the original author of a paper that's in the queue for peer replication supposed to wait for a year or two till the reviewer gets a beamtime on an appropriate measurement station? Even smaller: I did my PhD in a lab with a specific setup that only a single other group in the world had an equivalent to. You simply would not be able to replicate these results.

Peer replication is completely unfeasible in experimental fields of science. The current process of peer review is alright, people just need to learn that single papers standing by themselves don't mean too much. The "peer replication" happens over time anyway when others use the same tools, samples, techniques on related problems and find results in agreement with earlier papers.

Usually coming up with a idea is the easy part. For example, in my PhD project, i started with an idea from my advisor that he had in the early 2000.

Implementing the code for the simulation and analysis of the data? four months, at most. Running the simulation? almost three years until I had data with good enough resolution for publishing.

It’s also very easy to come up with bad ideas — I did plenty of that and I still do, albeit less than I used to. Finding an idea that is novel, interesting, and tractable given your time, skills, resources, and knowledge of the literature is hard, and maybe the most important skill you develop as a researcher.

For a reductive example, the idea to solve P vs NP is a great one, but I’m not going to do that any time soon!

I'd say it is a bad idea because you're not going to succeed at it.

Sure because that’s an easy example and kind of my point. When you’re starting your PhD, it can be hard to determine what questions you can and cannot answer given your skills and surrounding literature. That comes from experience and trying (and failing) a lot.

I think you don't understand how much work is involved in just building the techniques and expertise to pull some experiments off (let's not even talk about the equipment).

Even if someone meticulously documents their process, it could still take months to replicate the results.

I'm familiar with lithography/nanofabrication and I know that it is typically the case that a process developed in one clean-room can not be directly applied to a different clean room and instead one has to develop a new process based on what the other results.

Even in the same lab it can often happen that if you come back to a process after a longer time, that things don't work out anymore and quite a bit of troubleshooting ensues (maybe a supplier for some chemical changed and even though it should be the same formula it behaves slightly different).

Months. Haha.

I previously worked in agricultural research (in the private sector), and we spent YEARS trying to replicate some published research from overseas. And that was research that had previously been successfully replicated, and we even flew in the original scientists and borrowed a number of their PhD students for several months, year after year, to help us try to make it work.

We never did get it to fully replicate in our country. We ended up having to make some pretty extreme changes to the research to get similar (albeit less reliable) results here.

We never did figure out why it worked in one part of the world but not another, since we controlled for every other factor we could think of (including literally importing the original team's lab supplies at great expense, just in case there was some trace contaminant on locally sourced materials).

> We never did figure out why it worked in one part of the world but not another

Doesn't that indicate further research is needed? It sounds fascinating to me. (I know it isn't interesting for the people who couldn't get it working.) It also might indicate that the original research was incomplete in the sense that it might be a fluke due to specific conditions in the original country which isn't universal.

You are making an argument for replication not against it by stressing how meticulously documenting your process is an “if”, strewing how things like a supplier for a chemical can change and render reproduction impossible, and even if an observation can only be replicated in one clean room that means you effectively only have as long as that clean room remains opens to replicate it.

You are almost stressing all the ways we are producing garbage which is not science because it can never be reproduced due to deficient documentation of processes, changes in supply, and changes in the environment. All three can be minimized through peer replication. That’s superior to the ignoring that the science we’re doing is non-reproducible because it’s too hard and expensive approach.

Well there isn't a way to fix this replicative difficulty for many fields. So you're suggesting that the theoretical understanding gained from having produced a specific material just never be added to scientific understanding at all, ever. In an ideal world with ideal budget allotment there would be time and money to build entire replicating labs that can be moved and modularly reconfigured with absolute control over air particulates, pressure, flow, sufficient to replicate any room possible anywhere on earth. But if a lab in the himalayas was able to make a magnet that fundamentally reconfigures our understanfing of physical phenomena, we should simply abandon that potential deeper truth into were able to build a second lab in the Himalayas, or in the middle of the Atlantic.

Reproduction is hard, really really fucking hard. Just saying, that means we should replicate before trying to understand, means essentially cutting off the understanding.

And like others have said, if someone wants to build from it they'll depend on that information being correct, if no one can manage to ever build from it then the idea dies.

Also there's a huge difference between, replicate this study on infant response to stimulus, or spider colony behavior, and, replicate this incredibly intricate semiconductor that took years of configuration to correctly produce.

You can easily look up non peer reviewed papers on arxiv. Why would this be different with replication?