News for hackers. Stuff that matters.
Re:Ask HN: Teach me something new by ajb on Saturday April 30, 2022 @04:25AM (#31214327)
In statistics, the mean and median are both averages, but are technically quite different. The mean is the sum of all items divided by the number of them, while the median is the middle item in the sorted list. When they are different, this can have strange effects. For example, when estimating engineering tasks, the mean error is often positive while the median is often negative.

This is why your manager believes that engineers are too conservative and always pad estimates, while projects are always late. Indeed, they can learn both these things from observation without realising that they are contradictory. Because humans learn by the most common observation, which is represented by the median, but outcomes can be dominated by the less common, which affect the mean but hardly affect the median.

Re:Ask HN: Teach me something new by ZeroGravitas on Saturday April 30, 2022 @07:20AM (#31215118)
One of the rules of thumb of process management is to reduce the median, rather than the average time taken, because outliers are more likely to have one-off causes and so be harder to fix systematically. In fact, attempts to fix outliers can be a distraction and even net negative.
Re:Ask HN: Teach me something new by ajb on Saturday April 30, 2022 @08:31AM (#31215538)
Interesting. I wonder if that applies in SW dev though. In standard processes each correctly produced item satisfies a customer, but for sw dev individual tasks are internal and customers only care about the final outcome. Also standard processes normally involve Gaussian distributions, so large deviations are rare and can be called outliers, but software estimation has fat tailed or bimodal distributions, so large deviations are part of the standard order of business.
Re:Ask HN: Teach me something new by yshklarov on Saturday April 30, 2022 @12:01PM (#31216983)
> In statistics, the mean and median are both averages, but are technically quite different.

But numerically, they are not too wildly different! The difference between the mean and the median is at most one standard deviation. If the distribution is unimodal, then the difference is at most sqrt(3/5) standard deviations.

Re:Ask HN: Teach me something new by ajot on Saturday April 30, 2022 @09:44AM (#31216028)
> In statistics, the mean and median are both averages, but are technically quite different. The mean is the sum of all items divided by the number of them, while the median is the middle item in the sorted list.

There was an article posted here, one or two years ago, that explained how mean and median are very similar, as they are the values that minimize residuals or the squared of residuals.

I'm sorry I can't find it right now, I hope some other user knows which article I'm talking about and can link it down here, because it is, in my opinion, an excelent read.

Re:Ask HN: Teach me something new by BolexNOLA on Saturday April 30, 2022 @08:21AM (#31215487)
Interesting. I always learned mean/median/mode were different things. The mean is the average, the median is the middle value, and the mode is the largest value. I wasn’t aware of this issue with conflating mean/median.

That being said, I remember learning that and always wondering why I needed to know what “mode” was haha

Re:Ask HN: Teach me something new by leephillips on Saturday April 30, 2022 @09:32AM (#31215949)
The mode is the most common value, if it exists.
Re:Ask HN: Teach me something new by BolexNOLA on Saturday April 30, 2022 @12:35PM (#31217212)
Whelp that shows you how much I remember from middle school
Re:Ask HN: Teach me something new by GranBurguesa on Monday May 02, 2022 @04:31PM (#31240509)
i found the concept of https://en.wikipedia.org/wiki/Fr%C3%A9chet_mean helpful. it's a generalization of the different kinds of averages and you can express the arithmetic mean, geometric mean, harmonic mean, median or even mode just by swapping out the distance metric in the formula.
Re:Ask HN: Teach me something new by mirekrusin on Saturday April 30, 2022 @06:09AM (#31214762)
If the project is planned according to median, you accept up front that there is 50% chance the project will be late.
Re:Ask HN: Teach me something new by waqf on Saturday April 30, 2022 @07:21AM (#31215127)
It's worse than that. You need the median time for the whole project, which you can't get just from summing the medians of the individual tasks.
Re:Ask HN: Teach me something new by bigthymer on Saturday April 30, 2022 @11:02AM (#31216573)
There was a class in uni titled Robust Estimation that covers this.
Re:Ask HN: Teach me something new by leemailll on Saturday April 30, 2022 @09:49AM (#31216068)
median could be average, but that depends on the probability distribution of your data