Principal Component Projection Without Principal Component Analysis

Re:Principal Component Projection Without Principal Componen... by privong on Tuesday February 23, 2016 @10:19PM (#11164312)

A bit of a meta question, but what is the current thinking regarding the use of PCA in understanding data(sets)? My recollection is that PCA was somewhat en vogue in astronomy 8–10 years ago. I saw it applied to various mid-infrared data, but it seemed difficult to actually translate the principal components into useful physical knoweldge about the datasets or the astronomical objects. Since then, I rarely see astronomy papers with PCA analysis, and even then, the PCA analysis doesn't seem to contribute much to the physical understanding of the objects being studied.

Is this just a case of PCA being ill-suited to the analysis of these datasets in astronomy? Or is it a more general problem that PCA can reduce datasets to arbitrary component vectors but those vectors may not contain easily-quanitifiable physical information (but might contain predictive power, if an understanding of the underlying physical system is not the goal)?

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by sideshowb on Wednesday February 24, 2016 @06:12AM (#11165878)

The thing with PCA for dimensionality reduction in regression problems, specifically, is that it carries an assumption: that small variations in your predictor variables will not account for large variation in your dependent variable. This may or may not be justified. If you regard small variations as noise then by all means throw them away; if you don't, then you risk throwing out a minor component X that has a major influence on Y.

Regularized regression such as ridge regression can achieve the same thing without the risk of throwing away small but significant variation. So I think a current trend is to replace PCA+regression with ridge regression. This paper seems to take that trend a step further by replacing PCA itself with ridge regression even when the desired outcome is PCA, which is neat.

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by rcthompson on Tuesday February 23, 2016 @11:19PM (#11164546)

PCA and similar techniques are quite useful for exploratory data analysis in bioinformatics, in particular determining which experimental factors or batch effects have the largest influence on the data. Generally, if you can't identify your effect of interest in the first few principal components of a gene expression dataset, you're going to find few, if any, significant genes.

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by ucha on Tuesday February 23, 2016 @10:49PM (#11164423)

It's used in many quantitative trading models [1]. However, the time it takes compute the eigenvectors is irrelevant to the field because the stocks correlation matrix on which the which the PCA is applied is generally a long term correlation matrix that doesn't need to be updated too frequently.

[1] https://www.math.nyu.edu/faculty/avellane/AvellanedaLeeStatA...

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by thang on Wednesday February 24, 2016 @12:16AM (#11164710)

PCA is useful as a pre-processing step to reduce the size of a data set to be used as the input to a machine learning algorithm.

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by eggie5 on Wednesday February 24, 2016 @01:35AM (#11164962)

But then do you have to PCA all your input test data in the future too?

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by huac on Wednesday February 24, 2016 @01:42AM (#11164988)

the output of a PCA process is a system of linear combinations of the data. so yes, you'd take the principal components of your future inputs, but the process would be super fast (it's a linear transformation with known parameters) since you don't need to rerun anything.

you can also do online PCA (updating the model as you get new data) but i'm not sure about runtime/computation requirements.

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by eggie5 on Wednesday February 24, 2016 @02:45AM (#11165196)

I think the paper disagrees w/ you: "Computing principal components can be an expensive task..."

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by hiddencost on Wednesday February 24, 2016 @04:26AM (#11165534)

The result of PCA is a linear projection matrix. Applying that projection is fast.

You learn the projection from an initial dataset. That is "slow". You apply the projection to new data. That is "fast".

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by eggie5 on Wednesday February 24, 2016 @11:47AM (#11168100)

thanks for explaining

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by JupiterMoon on Wednesday February 24, 2016 @04:23AM (#11165522)

I think you misunderstood the OP. Once you have the PCs (and loadings) for a dataset converting new data into the same axis system is just a linear conversion.

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by cameldrv on Wednesday February 24, 2016 @01:24AM (#11164918)

One alternative to look at for some datasets is nonnegative matrix factorization. It's similar to PCA, but every element of each basis vector has to be positive. This tends to make the results a lot easier to interpret.

[ Parent | on HN ]

Re:Principal Component Projection Without Principal Componen... by wiennat on Wednesday February 24, 2016 @12:24AM (#11164730)

Basically, the purpose of PCA is to reduce the dimensions of the dataset by finding components that most influence the data. I think your last sentence is quite accurate to describe what PCA is.

[ Parent | on HN ]