News for hackers. Stuff that matters.
Re:Principal Component Analysis for Dummies by dnautics on Wednesday October 30, 2013 @08:13PM (#6644465)
doesn't the spread-out-ness then depend on the units? If your data have unit X on one axis, and unit Y on another, then how can you say that the "maximal-spread-outness" is in any given direction, when you can merely adjust the scale on one axis and alter how numerically spread out it looks?
Re:Principal Component Analysis for Dummies by Bill_Dimm on Thursday October 31, 2013 @12:20AM (#6645206)
Yes. The author of "Information Theory, Inference and Learning Algorithms" had this to say about it:

http://www.amazon.com/review/R16RJ2PT63DZ3Q/ref=cm_cr_rev_de...

Re:Principal Component Analysis for Dummies by joe_the_user on Thursday October 31, 2013 @01:16AM (#6645385)
An interesting argument.

It seems like PCA would already be a method that would only mean something if it was applied to comparable dimensions. What would transformed, dimensioned variables mean anyway? Chart A=mass - 3charge by B = mass + 2charge. What could a correlation mean.

Re:Principal Component Analysis for Dummies by twstws on Thursday October 31, 2013 @08:57AM (#6646490)
It's not dimensionally invalid if you remove the dimensions via normalization.
Re:Principal Component Analysis for Dummies by twstws on Wednesday October 30, 2013 @11:06PM (#6644991)
In most cases each variable is standardized to put them all on a comparable scale.
Re:Principal Component Analysis for Dummies by dnautics on Wednesday October 30, 2013 @11:23PM (#6645041)
normalizing to an arbitrary value still makes it arbitrary.
Re:Principal Component Analysis for Dummies by twstws on Thursday October 31, 2013 @08:52AM (#6646473)
Each variable is standardized to mean = 0, standard deviation =1. If you reject this as arbitrary, you are rejecting correlation analysis as a whole - this is exactly the same standardization done to two variables in bivariate correlation, extended to a multivariate data set.

PCA is a form of (or at least related to) correlation. With standardization the resulting transformation hihlights variables in the original data that are most highly correlated. Without standardization you're visualizing covariation. Unlike correlation, covariation is influenced by the magnitude of the variables.

By standardizing, you control for differences in the magnitude of the variables, and focus on their inherent variation instead.

Re:Principal Component Analysis for Dummies by dnautics on Thursday October 31, 2013 @02:17PM (#6648632)
1) this doesn't work for cases where your data are positive-definite.

However, let's set that aside. I apologize for being a bit obfuscatory. My point is: If this is the case, then the explanation in the OP is totally misleading, because your data shouldn't look like an ellipsoid, but rather a circle. PCA should only be used in situations where there is a reason to believe there is a mechanistically justifiable "hidden value" that underlies otherwise uncontrolled "independent variables", thus making a dimensional reduction reasonable.

This is not at all the situation that the OP goes over in the first part of the post.

Re:Principal Component Analysis for Dummies by twstws on Thursday October 31, 2013 @08:39PM (#6651098)
The example was a little clunky, but I don't find it misleading. A biplot of two normalized variables is elliptical, if the variables are correlated. This particular hand-drawn example does indeed look a bit weird, but that doesn't detract from the main point. It clearly shows the relationship between the original data and the ordination; it's a rigid rotation.

This is easily grasped with a 2d example, despite the fact that PCA makes no sense with only two variables.