If you measure 5, 5, 5, 9999999999, then your measured distribution is a peak at 5 and another at 9999999999. You can't say that 9999999999 is a surprising measurement unless you have a theoretical distribution - usually an implicit one - to evaluate at it and find a low probability. In other words, if being surprised means having your expectations violated, an expression of surprise implies that you must have had expectations to violate.

This argument doesn't make much sense to me. You can definitely associate a probability based just on the measured data. What if you measured 5, 5, 5, 5, 5, 5,5 (1000 more 5s) , 5, 999999999? I think it's pretty intuitive to say that 999999999 is unexpected relative to the 5s. And you certainly can have expectations that are generated purely based off of the measured distribution and not off of some theoretical model.

It is unexpected relative to the 5s, but it is also rare in your sample, meaning that its occurrence in the sample is in exact proportion to how often you'd expect it to occur, given the distribution you inferred from your sample. That means you have no grounds for singling it out as "abnormal."

And here we see the "Bayesian vs Frequentist" argument.

If this gets resolved in a HN thread you should publish it.

Obviously you can say that 9999999999 is very different from the rest of the sample consisting of 5 without an implicit distribution. It’s incredibly far from the median compared to all the other value.

But is it surprising to find it far from the median?

It doesn’t matter if it’s surprising. It might be a measurement mistake, a mixed population, a fat tail or a poor sample. It’s different therefore an outlier.

There are many statistical tests that will tell you that based on the observations alone (with no model). Basically this depends on how many 5s there are.

Every single one of those tests makes an implicit assumption about the distribution, explicit in its derivation. Usually, it is that the distribution is Gaussian.