News for hackers. Stuff that matters.

A comment on Latent Dirichlet Allocation Surprisingly Well Correlated w/ Google Rankings. Parent comment

Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by nl on Tuesday September 07, 2010 @02:21AM (#1668251)
I though it was kind of odd they did LDA rather than something more broadly used (eg LSA).

But I've never really looked at LDA, and Wikipedia says: Compared to standard latent semantic analysis which stems from linear algebra and downsizes the occurrence tables (usually via a singular value decomposition), probabilistic latent semantic analysis is based on a mixture decomposition derived from a latent class model. This results in a more principled approach which has a solid foundation in statistics. so maybe they made the right choice. (Not that I see what "solid foundation in statistics" really means in this context)

[ Parent | on HN ]
Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by cdavid on Tuesday September 07, 2010 @03:58AM (#1668364)
LSA is relatively similar in some abstract sense to the published pagerank algorithm. LDA is more powerful, in the sense that it can account for more complex relationships (but may be less accurate with large number of data - I have really no idea how those would scale and compare at google-like size).
[ Parent | on HN ]
Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by nl on Tuesday September 07, 2010 @07:08AM (#1668568)
Can you explain this some more?

My understanding of LDA is that it gives you document scores against queries based on the topics extracted using the LDA algorithm on the text in the page.

Pagerank, on the other hand scores based on external pointers (ie, references) but doesn't have anything to do with the text on the page.

[ Parent | on HN ]
Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by noelwelsh on Tuesday September 07, 2010 @09:22AM (#1668794)
It's the abstract sense that is important. Pagerank is a dimensionality reduction technique. It finds the first eigenvector of the transition matrix. Eigenvectors = PCA. LSI is basically PCA, but applied to the document-term matrix. LDA is a dimensionality reduction technique that makes use of more information.
[ Parent | on HN ]
Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by nl on Wednesday September 08, 2010 @07:00PM (#1673817)
Oh, I see.

I thought you were talking about some functional similarities, not the mathematical similarities.

[ Parent | on HN ]
Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by yread on Tuesday September 07, 2010 @04:25AM (#1668397)
For some applications Dirichlet mixtures are even better. I used this tool, which I can recommend, for deriving the parameters of the mixtures http://chasen.org/~daiti-m/dist/dm/
[ Parent | on HN ]
Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by jkan on Tuesday September 07, 2010 @05:05PM (#1670091)
I think "solid foundation in statistics" refers to the fact that LDA has a proper generative model of documents and topics, while LSA doesn't -- so that it's easier to reason about LDA and to build more complex models on top of it.
[ Parent | on HN ]
Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by equark on Tuesday September 07, 2010 @03:18AM (#1668327)
It means there is a likelihood function.
[ Parent | on HN ]