> Suppose we throw 1,000 baseballs, shoot one rocket (45 degree angle), record the distance each one flew, and present the 1,001 data points to someone for analysis.

But I would argue that's not really a case of an outlier, that's a mixture distribution because the underlying identifying feature (baseball or rocket) is missing.

You've almost grasped the point. Mixture distributions with a missing underlying feature are how outliers (in the meaningful, well-defined sense) are generated. And if you dig into this paper (I have not yet), I can almost guarantee you that this is foundational to the paper.

It's kind of funny just how long I had to work with statisticians to see that stats people and physics people look at generative processes in two completely different ways:

"There are two cultures in the use of statistical modeling to reach conclusions from data. One assumes that the data are generated by a given stochastic data model. The other uses algorithmic models and treats the data mechanism as unknown. The statistical community has been committed to the almost exclusive use of data models"

I can tell you from extensive experience that domain knowledge is more critical to understanding such issues than this algorithm. Fundamentally, such causes can not be identified if they don't exist in the data and the algorithm doesn't "see" them. Like the previous example where the identity of the object being hurled is hidden and all the data shows is the distance. And the causes change a lot from field to field.

I don't believe that your extensive experience tells you about the utility of an algorithm you've never used that was developed by Amazon researchers most likely to help with problems at Amazon.

> But I would argue that's not really a case of an outlier,

From the point of view of the data it is definitely an outlier.

> that's a mixture distribution because the underlying identifying feature (baseball or rocket) is missing.

This is the point of the paper. They literally say in the blog post summary:

To attribute the outlier event to a variable, we ask the counterfactual question “Would the event not have been an outlier had the causal mechanism of that variable been normal?”

Quantum mechanics allows a basketball to spontaneously morph into a rocket and back .. It's not very likely, but it can happen in theory. Statisticians should be able to deal with it in any case.

Then you require a hierarchial model which first estimates the membership of observations to discrete and non-overlapping sets of objects, and second estimates the quantity of interest. This is still a mixture distribution.

Statisticians are known to use this kind of model. See https://en.wikipedia.org/wiki/Dirichlet_process

i think robotics is probably an easier example to think about. if you have a sensor who's normal operation has a well defined gaussian, but every 2-4 seconds a spike in the data stream is seen for a millisecond due to any variety of reasons, you treat the spike as an outlier and dont let it ruin the gaussian that is actually governing your system, the spike is obviously not part of the nominal distribution, and it wouldn't be of any usable value to think of the system as a mixture.

This is treating the system as a mixture. It's just immediately getting rid of the component of the mixture you don't want. If you can do so reliably, you never have to think about it as a mixture again.

It's incredibly valuable to think of the system as a mixture, and work on both components of the problem in parallel. (Both increasing precision to narrow the width of the gaussian, and removing the defective interaction that causes the spikes.)

You are trying to add in a factor effect and call it an outlier. Its just unmodeled variation.

Are you sure you know the difference between an outlier and a point that is far out of the distribution of the other points because of unmodeled variation?

Just because you have a name for something doesn't mean it doesn't have other names as well.

Anyone who knows what they're doing would describe it as a probabilistic model, which necessarily includes likelihoods and confidence bounds with probabilities other than 0 or 1. There is a certain probability that it is a part of one distribution, and an inverse probability that it is part of another distribution. Statisticians worth their salt will not state certainties as you seem to demand.

Hmm, I'm not sure where I demanded a certainty?

> the difference between an outlier and a point that is far out of the distribution of the other points because of unmodeled variation

The phrasing implies mutual exclusivity here, which thus implies probability one of a data point belonging to one set and probability zero of that point belonging to the other set.

To be fair, in natural language it tends to be difficult to unambiguously represent ambiguity.

I wasn't drawing a dichotomy between "an outlier and a point that is far out of the distribution of the other points because of unmodeled variation". I was suggesting that they may be the same thing.