It took me a long time to realize that a principled reason for gaussian parametric distributions is the maximum entropy principle. Prior to that point, it had been presented as essentially arbitrary, even by established professors.
There seems to be an assumption that theoretical statistics is "too hard", and as a result there's a middle ground that gets left out. I haven't taught general stats courses in awhile (although I've taught advanced ones), but if I did, might start with Bregman divergences, and work down in the manner of your blog post.
I think there's an in-between that gets lost. You can teach principles without deriving long proofs of everything along the way.
Students don't get taught the underling principles and philosophies to choose from, and I think this leads to the "holy grail" issues you're referring to.
It seems to be changing a bit with new interest in Bayesian methods, but that's just the tip of the iceberg.
Humans want to distill vast amounts of information to a more manageable amount, like for example a single number.
Equity analysts look at accounting metrics, psychologists look at psychometrics test cores, doctors look at some function of blood pressure, etc etc.
Any person with deductive and sceptical mental faculties in place, will recognize that these are all simplifications, and cannot be used to deliver a unified truth.
Also, being aware of this has very little to do with being technical or not (for example, plenty of programmers only look at a CPU's clock speed to gauge performance).
Anyhow, nice post.
Well yes, that is their purpose.
What happens when you minimize Ln, with n > 2? Why don't we use any of those?
A single-number statistic is _going_ to leave things out, so if you must boil things down into one number, or even a few numbers, you're going to lose something that you had in the raw data. This is why I find claims along the lines of "statistics don't tell the whole story" a little bemusing - of course they don't, the very definition of a statistic is a summarization of data that is easier to work with. The question is what data is kept or lost, or more generally what importance we place on different aspects of the raw data such that it's reflected in our descriptive statistics.
The lessons for non-technical people who want to communicate with descriptive statistics are to recognize that summarization is inherent in the nature of any descriptive statistic, that they are thereby opinionated in some way in terms of what they've preserved and what they've left out, and to recognize whether those opinions are appropriate for your purpose.