News for hackers. Stuff that matters.
Re:Principle Component Analysis by tel on Friday August 17, 2012 @04:18PM (#4398577)
PCA goes far deeper than meets the eye. For instance, it's a well-known phenomenon that too much dimensionality can actually drive predictor performance to random, but PCA can mitigate that. It's a basically the bread and butter of practical unsupervised learning.
Re:Principle Component Analysis by mturmon on Friday August 17, 2012 @05:02PM (#4398782)
"bread and butter of practical unsupervised learning" -- true, although I might have said "exploratory data analysis".

If you can make a vector out of it somehow, it can't hurt to try PCA. Because you don't have to figure out some fancy tailored model, or really (cough, cough) understand much about the data at all. (It sounds like I'm being sarcastic, but I'm serious -- sometimes all you want is a quick look.)

Unsupervised clustering is a similar technique.

Re:Principle Component Analysis by tel on Friday August 17, 2012 @07:29PM (#4399386)
I find that more often than expected, PCA (or maybe MDS) gets a majority of the performance of any kind of unsupervised method. If you're really interested in exploring the data and methodologies, then PCA is a poor stopping point... but if you just want it to work, it's surprising how well PCA tradeoffs are good tradeoffs.

All the obvious caveats apply to that whole line of thought, though.

Re:Principle Component Analysis by snikolov on Friday August 17, 2012 @05:26PM (#4398901)
Indeed, projection onto the principal subspace is a kind of regularization, which makes for better generalization.

This kind of makes sense intuitively. These slides go into more detail (the whole course is great), where it says "Projection regularizes!"

http://www.mit.edu/~9.520/spring09/Classes/class07_spectral....