News for hackers. Stuff that matters.
Re:Principle Component Analysis by mturmon on Friday August 17, 2012 @11:07PM (#4400071)
Thanks for the reminder. I was thinking of this 1991 paper, which I ran into a long time ago:

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.41.1...

They used a (linear) "neural network" with gradient descent training that implemented PCA (kind of an iterative graham-schmidt process), and got Gabor-like filters. I think a lot of people have done similar experiments, with varying results.

Re:Principle Component Analysis by psb217 on Sunday August 19, 2012 @09:06PM (#4405859)
I hadn't seen that paper before; thanks for the reference. I read through it and saw that they were reweighting the sampled image patches with a Gaussian mask prior to learning, which explains how they got Gabor-like filters. The masking effectively forced the learned filters to have localized receptive fields, while locality/nonlocality is generally one of the (visually) clearer differences between filters learned with ICA/PCA.

In other words, the Gaussian-modulated part of Gaussian-modulated sinusoids was built into their learning process, rather than appearing as an emergent property. I also chuckled a bit when they described how computing eigenvectors for 4096x4096 matrices was "beyond reasonable computation".