Re:A polynomial autoencoder beats PCA on transformer embeddings
by ComplexSystems on Saturday May 09, 2026 @04:37AM
(#48073206)
What makes this different from just kernel PCA with the quadratic kernel?
[ Parent
| on HN ]
Re:A polynomial autoencoder beats PCA on transformer embeddings
by pleshkov on Sunday May 10, 2026 @07:56AM
(#48083231)
It's close but not the same. Kernel PCA lifts all D coordinates which gives M around 525k at D = 1024. In the post I do PCA first to reduce D to d = 256, then lift only those d coordinates, M = 33k. Much smaller, much faster Ridge solve.
Re:A polynomial autoencoder beats PCA on transformer embeddings
by ComplexSystems on Sunday May 10, 2026 @02:41PM
(#48086568)
That makes sense. If you could magically just get the top d PCs in quadratic kernel space without having to compute the whole kernel matrix, and then just do top-d quadratic PCs -> ridge, would that be better than doing the PCA -> top-d -> quadratic kernel ridge as you are now?