News for hackers. Stuff that matters.
Re:A short statistical reasoning test by usgroup on Monday August 18, 2025 @05:27AM (#44938779)
From the question:

“However, it is very important that the uncertainty in the number of trials is taken into account because over-estimating a fraction is a costly mistake.“

Seems fairly clear to me that you’re supposed to use a lower bound estimate to take into account variance on the fraction due to the number of trials in a way to bounds the chance of over estimation.

Further, there is no need for a heuristic when there a several statistical models for this exact problem with clear properties. Some are given in the answer.

Re:A short statistical reasoning test by thekoma on Monday August 18, 2025 @05:41AM (#44938851)
Out of context, the expression "the uncertainty in the number of trials" would refer to missing knowledge in terms of how many trials actually ran.

In the context of the post this doesn't make sense, so the reader is left to hypothesize what the writer actually meant.

Re:A short statistical reasoning test by usgroup on Monday August 18, 2025 @06:47AM (#44939230)
I agree it could be clearer but as a general rule, if you find an interpretation under which the question doesn’t make sense, try considering another interpretation.
Re:A short statistical reasoning test by taylorius on Monday August 18, 2025 @05:42AM (#44938853)
I think "uncertainty due to the number of trials" would be clearer.
Re:A short statistical reasoning test by tomsmeding on Monday August 18, 2025 @06:34AM (#44939140)
Uncertainty in the number of successes due to the number of trials, would be even better.
Re:A short statistical reasoning test by jdhwosnhw on Monday August 18, 2025 @07:56AM (#44939644)
That would also be an incorrect phrasing. This entire thread is a good illustration of the difficulty of speaking precisely about probabilistic concepts.

(The number of successes has zero uncertainty. If you flip a coin 10 times and get 5 heads, there is no uncertainty on the number of heads. In general, for any statistical model the uncertainty is only with respect to an underlying model parameter - in this example, while your number of successes is perfectly known, it can be used to infer a probability of success, p, of 0.5, and there is uncertainty associated with that inferred probability.)

Re:A short statistical reasoning test by jldugger on Monday August 18, 2025 @12:07PM (#44942188)
I guess I struggle to understand why under estimation is worse than over estimation, when the final result is a ranking. It seems like they're equally likely to produce an incorrect ranking!