Latent Dirichlet Allocation Surprisingly Well Correlated w/ Google Rankings

Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by nl on Tuesday September 07, 2010 @07:08AM (#1668568)

Can you explain this some more?

My understanding of LDA is that it gives you document scores against queries based on the topics extracted using the LDA algorithm on the text in the page.

Pagerank, on the other hand scores based on external pointers (ie, references) but doesn't have anything to do with the text on the page.

[ Parent | on HN ]

Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by noelwelsh on Tuesday September 07, 2010 @09:22AM (#1668794)

It's the abstract sense that is important. Pagerank is a dimensionality reduction technique. It finds the first eigenvector of the transition matrix. Eigenvectors = PCA. LSI is basically PCA, but applied to the document-term matrix. LDA is a dimensionality reduction technique that makes use of more information.

[ Parent | on HN ]

Re:Latent Dirichlet Allocation Surprisingly Well Correlated ... by nl on Wednesday September 08, 2010 @07:00PM (#1673817)

Oh, I see.

I thought you were talking about some functional similarities, not the mathematical similarities.

[ Parent | on HN ]