Re:Claude Fable 5.2 and Mythos 5.2
by jerkstate on Monday November 09, 2026 @03:48PM
(#50431880)
from my basic stats knowledge, it's not reliable to calculate the confidence interval of the sample proportion with fewer than about 10 samples on both sides. So if you're only seeing 1 or 2 failures in 20, and that approximately held as you scaled, you would have to run about 200 tests with each in order to get at least 10 failures in order to calculate the confidence intervals of the two samples and see if they overlap or not.
[ Parent
| on HN ]
Re:Claude Fable 5.2 and Mythos 5.2
by disgruntledphd2 on Monday November 09, 2026 @04:03PM
(#50432094)
And this is why not many people do this. Even if you have the decals and the statistical knowledge, it's pretty expensive to do this regularly.