Typical trials are designed to have a large enough sample size that they will be able to detect any effect big enough to be clinically important.
Here, they just went with the sample size they had available (understandably), which was so small that only a very large effect could be clearly established. It is completely wrong to interpret the lack of a statistically significant effect with this small sample size as showing that the drug doesn't work.
Consider the result regarding transfer to ICU or death:
20.2% patients in the HCQ group were transferred to the ICU
or died within 7 days vs 22.1% in the no-HCQ group (16 vs 21
events, relative risk [RR] 0.91, 95% CI 0.47-1.80)
Again, note also that more patients on HCQ were diagnosed with ARDS, so if you pick a different measure, you get the opposite result.
Adding to this, there are at least two reasonably sized ongoing trials of HCQ with appropriate randomization and blinding in North America alone (I believe there are a number of others elsewhere). I would encourage everyone to withhold judgement until either they post results, or a similarly reliable alternative data source appears.
Target size of 3000 (https://clinicaltrials.gov/ct2/show/NCT04308668).
Target size of 400 (https://clinicaltrials.gov/ct2/show/NCT04329923).
(It was also mostly junk science of course too)
> For example, it's highly unlikely (roughly less than 2.5%) that HCQ reduces the onset of ARDS by half.
is not one that can be justified by frequentist statistics such as those used in the article. We can't make probabilistic claims like that unless we go Bayesian.
Well-designed drug trials typically plan in advance to have the sample size necessary to detect an effect of clinical significance. If, say, a 20% reduction in ARDS would be valuable, you would work out what sample size you'd need to reliably detect such a reduction. In this case, we simply don't have the sample size to conclude much of anything, because the confidence intervals include what I (as a non-physician, admittedly) see as a pretty wide range of clinically meaningful effect sizes, and we can't distinguish between them.
In frequentist statistics, the true population parameter is fixed, not random, so the claim "the relative risk will be less 0.65 2.5% of the time" has no meaning. The 95% claim is that if we repeatedly sample new data, 95% of the time we will generate a confidence interval covering the true parameter; but that does not say anything about the individual confidence interval in this specific case, about which we have no probability claim. Indeed, it's not clear what a 95% chance would mean for a specific interval, since probability in frequentist statistics is defined in terms of long-run probabilities over repeated samples of data.
But the impulse to interpret confidence intervals in a Bayesian way is common, since it's the question we want confidence intervals to answer. You can find a lot of academic papers arguing about this as a sign of the inadequacy of frequentist statistics.
For example, it's highly unlikely (roughly less than 2.5%) that HCQ reduces the onset of ARDS by half.
If it ultimately turns out that HCQ reduces the rate of death by 5%, we'd have to do a tremendously large trial to distinguish that effect from the control with any confidence that it's not actually killing 2% more people. Additionally, you'd have to consider that vs. the fact that it will kill some fraction of people who take it from its cardiac side effects, even if doctors are vigilant about discontinuing therapy when arhythmias are detected.