News for hackers. Stuff that matters.
Re:Study: Chances of serious side effects from coronavirus v... by nerdponx on Tuesday September 06, 2022 @04:04PM (#32742516)
I actually went and plugged in the numbers for coagulation disorders into a frequentist likelihood ratio test for a difference in proportions, and got p = 0.23.

Feel free to check my work (in R):

    coag_vax <- 16
    p_vax <- 8.5 / 10000
    num_vax <- round(coag_vax / p_vax)
    cat("#(vaccine) = ", num_vax, "\n", sep = "")
    cat("#(coagulation disorder | vaccine) = ", coag_vax, "\n", sep = "")
    cat("P(coagulation disorder | vaccine) = ", p_vax, "\n", sep = "")
    cat("\n")

    coag_plac <- 10
    p_plac <- 5.3 / 10000
    num_plac <- round(coag_plac / p_plac)
    cat("#(placebo) = ", num_plac, "\n", sep = "")
    cat("#(coagulation disorder | placebo) = ", coag_plac, "\n", sep = "")
    cat("P(coagulation disorder | placebo) = ", p_plac, "\n", sep = "")
    cat("\n")

    coag_same <- coag_vax + coag_plac
    num_same <- num_vax + num_plac
    p_same <- coag_same / num_same
    cat("#(null) = ", num_same, "\n", sep = "")
    cat("#(coagulation disorder | null) = ", coag_same, "\n", sep = "")
    cat("P(coagulation disorder | null) = ", p_same, "\n", sep = "")
    cat("\n")


    ## Frequentist likelihood ratio test that the proportions are identical

    # https://stats.stackexchange.com/a/373448/36229
    # http://people.musc.edu/~bandyopd/bmtry711.11/lecture_02.pdf
    l_same <- dbinom(coag_vax, num_vax, p_same) * dbinom(coag_plac, num_plac, p_same)
    l_diff <- dbinom(coag_vax, num_vax, p_vax) * dbinom(coag_plac, num_plac, p_plac)
    t <- -2.0 * log(l_same / l_diff)
    p <- 1 - pchisq(t, 1)
    cat("LR test statistic = ", t, "\n", sep = "")
    cat("LR test p-value = ", p, "\n", sep = "")
Output:

    #(vaccine) = 18824
    #(coagulation disorder | vaccine) = 16
    P(coagulation disorder | vaccine) = 0.00085

    #(placebo) = 18868
    #(coagulation disorder | placebo) = 10
    P(coagulation disorder | placebo) = 0.00053

    #(null) = 37692
    #(coagulation disorder | null) = 26
    P(coagulation disorder | null) = 0.0006898015

    LR test statistic = 1.412184
    LR test p-value = 0.2346942
And for a 95% CI around the difference in proportions, I got [0, 8.5]:

    # https://pubmed.ncbi.nlm.nih.gov/25163425/
    # var_vax <- num_vax * p_vax * (1 - p_vax)
    # var_plac <- num_plac * p_plac * (1 - p_plac)
    var_pooled <- (p_vax * (1 - p_vax) / num_vax) + (p_plac * (1 - p_plac) / num_plac)
    std_pooled <- sqrt(var_pooled)
    ci_lo <- (p_vax - p_plac) - qnorm(0.975) * std_pooled
    ci_hi <- (p_vax - p_plac) + qnorm(0.975) * std_pooled
    cat("95% CI (Gaussian approximation) = [ ", max(ci_lo, 0.0) * 10000, " in 10k , ", min(ci_hi, 1.0) * 10000, " in 10k ]\n", sep = "")
Output:

    95% CI (Gaussian approximation) = [ 0 in 10k , 8.502481 in 10k ]
I also ran it through a completely-un-tuned Bayesian model and got a similarly inconclusive result:

    library(brms)

    coag_data <- data.frame(
      category = c("vaccine", "placebo"),
      n_sample = c(num_vax, num_plac),
      n_coag = c(coag_vax, coag_plac))

    fmla <- n_coag | trials(n_sample) ~ category

    prior <- get_prior(fmla, coag_data, family = binomial)
    cat("Prior:\n")
    print(prior)

    model <- brm(
      fmla,
      data = coag_data,
      family = binomial(link = "logit"),
      prior = prior,
      iter = 1000)
    cat("Model:\n")
    print(model)
Output (truncated):

    Population-Level Effects:
                    Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
    Intercept          -7.58      0.32    -8.23    -6.98 1.00     1000     1278
    categoryvaccine     0.49      0.41    -0.27     1.32 1.00     1319     1239

    Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
    and Tail_ESS are effective sample size measures, and Rhat is the potential
    scale reduction factor on split chains (at convergence, Rhat = 1).
Conclusion: It's suggestive that the vaccine is more dangerous than the placebo but not at all a strong confirmation.

I did skim back over the paper and it looks like they categorized "serious" in such a way that maybe hospitalization is not a terrible baseline for comparison. But it's still pretty questionable that the incidence of "serious" coagulation disorders in the placebo group is higher than the incidence of hospitalization in the placebo group, and it makes me think that they're making an invalid comparison there. So this is an inconclusive result, presented in what is at best a hand-wavey framework.

There are no comments on this comment yet.