News for hackers. Stuff that matters.
Re:Lesson of ivermectin: meta-analyses based on summary data... by gnicholas on Wednesday September 22, 2021 @12:15PM (#28618154)
Do researchers ever pre-register their studies when they do meta analyses? Pre-registration makes sense for the timing of doing actual studies, but not as much in the case of meta analyses (which presumably happen on a much faster timeline).

But given what we've seen with p-hacking, it seems like it could be good if people pre-registered their meta analyses before crunching the numbers.

Re:Lesson of ivermectin: meta-analyses based on summary data... by nradov on Wednesday September 22, 2021 @04:32PM (#28621391)
Sure some researchers do pre-register their meta analysis studies on clinicaltrials.gov. It's generally optional. Of course there's nothing stopping a dishonest researcher from doing a little secret p-hacking before pre-registering.
Re:Lesson of ivermectin: meta-analyses based on summary data... by mohammad_ali85 on Wednesday September 22, 2021 @03:51PM (#28620957)
For systematic reviews there are one of two ways that I'm aware of: PROSPERO [1] and Cochrane[2]. Cochrane is slightly more rigorous for conducting a review of this type. Although you do declare what meta-analyses you intend to do, you can't be overly explicit as it is largely dependent on the number of studies that are returned in your search strategy. For example two studies is too small for a meta-analysis so you would do a narrative review (discuss what you found).

[1]https://www.crd.york.ac.uk/prospero/

[2]https://www.cochranelibrary.com

Re:Lesson of ivermectin: meta-analyses based on summary data... by rscho on Wednesday September 22, 2021 @04:43PM (#28621522)
Good practices dictate that preregistration should always be done nowadays, including for meta-analysis. In practice though, this has very little impact. There is absolutely no guarantee that what's written in the paper is a faithful report of what really happened. There's nothing holding back dishonest (or, at that point I guess I should write 'standard') researcher from doing absolutely whatever they want.
Re:Lesson of ivermectin: meta-analyses based on summary data... by ekianjo on Wednesday September 22, 2021 @08:17PM (#28623772)
how would you know when they actually started the analysis?
Re:Lesson of ivermectin: meta-analyses based on summary data... by gnicholas on Wednesday September 22, 2021 @10:46PM (#28624732)
That's one of the reasons I was asking. Perhaps they could pre-register prior to being given the full data from other studies?
Re:Lesson of ivermectin: meta-analyses based on summary data... by ekianjo on Saturday September 25, 2021 @12:06AM (#28650434)
If the data is kind of public or going around in your circles you could easily hack this pre-registration constraint.