Re:Median and Mad Revisited with an Online Estimator
by AnthonyLloyd on Thursday April 09, 2020 @04:33AM
(#22820770)
Happy to answer questions if anything is not clear or discuss.
[ Parent
| on HN ]
Re:Median and Mad Revisited with an Online Estimator
by Lemaxoxo on Thursday April 09, 2020 @08:55AM
(#22822050)
Would you happen the memory complexity of this algorithm? I maintain an online machine learning library written in Python, called creme, where we implement online statistics. We have a generic onine algorithm for estimating quantiles, and so a specific algorithm for estimating medians would be welcome. I'm always on the lookout for such online algorithms.
Re:Median and Mad Revisited with an Online Estimator
by AnthonyLloyd on Thursday April 09, 2020 @09:09AM
(#22822152)
It's a fixed memory size but gets released when you move into the recursive part. Increasing the fixed part will give you a better starting estimate.
See here for quantiles: https://stackoverflow.com/questions/1058813/on-line-iterator...
The P^2 algorithm used in creme is interesting. For the median it would give a 2 sided median deviation. Maybe this could be changed slightly to be symmetric and give Median and MAD. I'll look into.
Re:Median and Mad Revisited with an Online Estimator
by johnmyleswhite on Thursday April 09, 2020 @12:51PM
(#22824513)
Can you link to a proof of the equation starting at "The standard error of the sample Median approximates to"?
Re:Median and Mad Revisited with an Online Estimator
by AnthonyLloyd on Thursday April 09, 2020 @01:15PM
(#22824820)
It's a quite hand wavy central limit theory for large N or close to normal distribution:
Re:Median and Mad Revisited with an Online Estimator
by lonelappde on Thursday April 09, 2020 @08:59AM
(#22822078)
What is MAD?
Re:Median and Mad Revisited with an Online Estimator
by AnthonyLloyd on Thursday April 09, 2020 @09:04AM
(#22822109)
Median Absolute Deviation
Discussed in the previous post: http://anthonylloyd.github.io/blog/2016/10/21/MAD-Outliers
It's roughly the range around the median where half the sample is within (since you can do 2 sided MAD also).