News for hackers. Stuff that matters.
Re:Jepsen: Datomic Pro 1.0.7075 by cdchn on Wednesday May 15, 2024 @08:10PM (#40373850)
"Absence of evidence is not evidence of absence."
Re:Jepsen: Datomic Pro 1.0.7075 by andreareina on Wednesday May 15, 2024 @09:19PM (#40374260)
If you've looked, it is. The more and the better you look, the better evidence it is.
Re:Jepsen: Datomic Pro 1.0.7075 by kelseyfrog on Wednesday May 15, 2024 @09:50PM (#40374456)
If you run it through bayes theorem, it adjusts the posterior very little.
Re:Jepsen: Datomic Pro 1.0.7075 by cwillu on Thursday May 16, 2024 @03:22PM (#40382092)
If a test almost always finds something, then the failure of that test to find something is strong evidence.
Re:Jepsen: Datomic Pro 1.0.7075 by kelseyfrog on Thursday May 16, 2024 @04:06PM (#40382543)
I'd be happy to see you numbers for estimated likelihood, prior, and marginal probability if you have them. I'm curious what you get.
Re:Jepsen: Datomic Pro 1.0.7075 by kelseyfrog on Wednesday May 15, 2024 @09:49PM (#40374447)
Thank you. I've updated my initial guess of p(critical bugs | did not find critical bugs) from 0.5 to 0.82 given my estimate of likelihood and base rates.
Re:Jepsen: Datomic Pro 1.0.7075 by nine_k on Thursday May 16, 2024 @02:25PM (#40381556)
s/evidence/proof/.

Evidence of absence ("we searched really carefully and nothing came up") does update the Bayesian priors significantly, so the probability of absence of bugs can now be estimated as much higher.