Re:New method identifies the root causes of statistical outl...
by ubbr on Tuesday July 19, 2022 @02:31PM
(#32155964)
Why focus on outliers and not all quantitative causal relationships in the graph like "tell me what causes X"?
[ Parent
| on HN ]
Re:New method identifies the root causes of statistical outl...
by kqr on Tuesday July 19, 2022 @02:47PM
(#32156174)
Because there's nothing in particular that causes what's happening in the central part of the distribution. The centrally located outcomes are the result of combining a myriad of small factors this way and that way. Outliers usually have an identifiable, single cause.
This is why in statistical process control these types of outcome are known as "common-cause variation" and "assignable-cause variation".
Re:New method identifies the root causes of statistical outl...
by jazzyjackson on Tuesday July 19, 2022 @03:15PM
(#32156554)
I think it would be very useful to look at an outlier and know if it's a legitimate measurement that should be kept in the dataset or an aberration that can be safely removed from the dataset.
I am not a statistician, so I don't know under what circumstances outliers are usually thrown out.
Re:New method identifies the root causes of statistical outl...
by kqr on Tuesday July 19, 2022 @04:33PM
(#32157580)
> I am not a statistician, so I don't know under what circumstances outliers are usually thrown out.
As an industrial statistician, I can tell you: way too often.
Outliers are the signal among the noise. They indicate something. It is nearly always worth finding out what, instead of removing them. If they indicate a flaw with measurement or the process, then fix that flaw and re-do the measurement or re-run the process. Outlier gone! But in a much more informative way.
Re:New method identifies the root causes of statistical outl...
by mcswell on Tuesday July 19, 2022 @05:04PM
(#32157955)
I have absolutely no knowledge of this stuff (in particular of your industry), but I would have thought that most outliers indicate either a measurement error (which you suggest) or a cause that hasn't been taken into account. For the latter, I'm thinking for example about countries whose life expectancy is completely out of line with their per capita income, or some such. Do unexpected causes appear to cause outliers in your field?