News for hackers. Stuff that matters.
Re:Text Matching Using Cosine Similarity by nerdponx on Thursday December 27, 2018 @10:47AM (#18770115)
The link goes even deeper than that. Cosines have an intimate relationship with Euclidean distance, and the fundamental statistics concept of variance is in turn intimately related to Euclidean distance... and Gaussian distributions (perhaps the most continuous distribution family because of the Central Limit Theorem) are parameterized directly by mean and variance. And Gaussian models have a convenient habit of reducing to straightforward linear algebra as a result. The fact that Gaussian problems also happen to be nicely differentiable is yet another bonus. Oh, and least-squares linear regression (read: fit a line to optimize the Euclidean distance between your prediction vector and the data) is equivalent to a Gaussian maximum likelihood model.

Basically, everything boils down to high-school trigonometry, because it's the most natural way to define distances in our world. I still marvel at it.

Re:Text Matching Using Cosine Similarity by linearsep on Thursday December 27, 2018 @04:35PM (#18772817)
> The link goes even deeper than that. Cosines have an intimate relationship with Euclidean distance, and the fundamental statistics concept of variance is in turn intimately related to Euclidean distance... and Gaussian distributions (perhaps the most continuous distribution family because of the Central Limit Theorem) are parameterized directly by mean and variance. And Gaussian models have a convenient habit of reducing to straightforward linear algebra as a result. The fact that Gaussian problems also happen to be nicely differentiable is yet another bonus. Oh, and least-squares linear regression (read: fit a line to optimize the Euclidean distance between your prediction vector and the data) is equivalent to a Gaussian maximum likelihood model.

Thanks for this insight! Can you also suggest a book or other materials that I can read to understand this in greater depth?

Re:Text Matching Using Cosine Similarity by nerdponx on Saturday December 29, 2018 @09:12AM (#18783615)
Note that I meant to write

perhaps the most important continuous distribution family because of the Central Limit Theorem

instead of

perhaps the most continuous distribution family because of the Central Limit Theorem

As for a single book, not really. See my response to the sibling comment for some more insight.

Re:Text Matching Using Cosine Similarity by mlevental on Thursday December 27, 2018 @08:02PM (#18774060)
I really hate this kind of numerology. cosines don't have intimate relationship with euclidean distance - the dot product and the norm aren't equivalent https://math.stackexchange.com/questions/528864/is-every-nor.... I also have no idea what you mean by normal distribucións being most continuous because CLT? and then what's linearity got to do with continuity? normal distribucións are convenient to work with simply because of the exponential (other exponential families are also nice to work with). again don't know what you mean by normals being nicely differentiable. lots of PDFs are differentiable (in fact most functions are easy to differentiate). finally you only have to farther than an undergrad book to see what most things are actually not high school trig (e.g. time dependent models like SDEs or Bayesian inference with intractable marginals)
Re:Text Matching Using Cosine Similarity by nerdponx on Saturday December 29, 2018 @09:19AM (#18783652)
I'm not talking in generalities. I'm talking about the standard inner product space on R^n that most people use (either implicitly or explicitly) every day.

You are right that the normal distribution is not continuous because of the CLT. That was a typo. I meant "most important continuous" distribution.

Yes, other exponential distributions are convenient to work with, but the Gaussian maximum-likelihood estimate has a closed-form expression defined with just linear algebra. That's really damn nice in my opinion.

The point is that distance (in most real-world applications) is equivalent to the length of the hypotenuse of a right triangle in Euclidean space. So in my opinion, yes, high-school trigonometry governs a lot of advanced mathematics, including in statistics and machine learning.

Call it numerology if you want, I'm just trying to get people excited about math here.