News for hackers. Stuff that matters.
Re:A high-dimensional sphere spilling out of a high-dimensio... by CrazyStat on Monday January 17, 2022 @02:45PM (#29970889)
No. Mahalanobis distance is not an alternative to Euclidian distance because it's not even measuring the same kind of distance. The are incommensurate, both figuratively and literally: Mahalanobis distance is unitless while Euclidian distance is not.

Euclidian distance measures the distance between two points, while Mahalanobis measures the distance between a distribution (canonically multivariate normal) and a point. Mahalanobis distance is not a generalization if Euclidian distance, it's an altogether different concept of distance that doesn't even make sense without talking about a distribution with mean and covariance matrix.

Re:A high-dimensional sphere spilling out of a high-dimensio... by srean on Monday January 17, 2022 @03:54PM (#29971586)
I agree about your larger relevant point but the following that you say is bit of a red herring

> Euclidean distance measures the distance between two points, while Mahalanobis measures the distance between a distribution (canonically multivariate normal) and a point

In a discussion about metric and metric spaces we dont care about those things, its abstracted out and considered irrelevant. All that matters is that we have a set of 'things' and a distance between pairs of such things that satisfies the properties of being a distance (more precisely, properties of being a metric).

@CrazyStat (I cannot respond to your comment so leaving it here)

I think you overlooked

> things that satisfies the properties of being a distance (more precisely, properties of being a metric).

that I wrote. Of course it has to satisfy the properties of being a metric. The red herring, as far as dimensionality is concerned, is the complaint that Mahalanobis is defined over distributions while Euclidean is over points.

The part about MD that you get absolutely right is its nothing but Euclidean distance in a space that has been transformed by a linear transformation. MD (the version with sqrt applied) and ED aren't that different, especially so in the context of dimensionality

@CrazyStat response to second comment.

It indeed isnt, its just Euclidean distance under linear transformation. I was just quoting you, you had said

> while Mahalanobis measures the distance between a distribution

My point was even it is defined for distributions its not really relevant.

> Mahalanobis "distance" is more closely related to a likelihood function than to a true distance function.

That's a subjective claim, and open to personal interpretation. Mathematically MD is indeed a metric (equivalently a distance) and it does show up in the log likelihood function. Mahalanobis was a statistician, but MD is a bonafide distance in any finite dimensional linear space, with possible extensions to infinite dimensional spaces by way of a positive definite kernel function (or equivalently, the covariance function of a Gaussian process)

Re:A high-dimensional sphere spilling out of a high-dimensio... by CrazyStat on Monday January 17, 2022 @04:02PM (#29971662)
A function that measures the distance between two different classes of 'things' (distribution and point, in this case) is necessarily not a distance metric. It trivially fails to satisfy the triangle inequality, because at least one of d(x,y), d(x,z), d(y,z) will be undefined--no matter how you choose x, y, z you'll end up either trying to measure the distance between two points or the distance between two distributions, neither of which can be handled.

This is not a red herring, it's a fundamental issue.

Re:A high-dimensional sphere spilling out of a high-dimensio... by CrazyStat on Monday January 17, 2022 @04:27PM (#29971865)
MD isn't defined over distributions, though. There are perfectly good distance metrics defined over distributions, but MD isn't one of them. It's a "distance" between one distribution and one point, not between two distributions or two points.

Mahalanobis "distance" is more closely related to a likelihood function than to a true distance function.

Re:A high-dimensional sphere spilling out of a high-dimensio... by anon_123g987 on Monday January 17, 2022 @03:19PM (#29971255)
What has more seeds, an apple or a fruit?
Re:A high-dimensional sphere spilling out of a high-dimensio... by CrazyStat on Monday January 17, 2022 @03:23PM (#29971299)
What's a better fruit, an apple or an apple pie?

Like Mahalanobis distance, apple pie is not a fruit and is not a generalization of an apple.