Re:Reproducibility trial: 246 biologists get different resul...
by rrsp on Saturday October 14, 2023 @05:49AM
(#37879274)
Does anyone know if there’s a reason why we’ve not agreed on a standard way of analysing data for any given type of study or type of outcome?
Maybe it’s naive but my intuition would be that if someone were conducting a randomised control trial with a hypothesis that A > B then there should be a known best practice for analysing that data to check the hypothesis. A way that is reproducible, otherwise surely it’ll become a shitfest of people trying multiple methods of analysis and then publishing whichever produced the ‘best’ result.
> The researchers gave scientist-participants one of two data sets and an accompanying research question: either “To what extent is the growth of nestling blue tits (Cyanistes caeruleus) influenced by competition with siblings?” or “How does grass cover influence Eucalyptus spp. seedling recruitment?”
What follows is that the variations in choices of how the data is analyzed gives a wide range of 'strengths' that when averaged is not significant, or extreme and opposing individual strength results. What this is highlighting is that the data analysis (even for identically reproduced data) is a source of irreproducibility meaning that it itself needs to be explicit and validated.