News for hackers. Stuff that matters.
Re:A polynomial autoencoder beats PCA on transformer embeddings by ComplexSystems on Saturday May 09, 2026 @04:37AM (#48073206)
What makes this different from just kernel PCA with the quadratic kernel?
Re:A polynomial autoencoder beats PCA on transformer embeddings by pleshkov on Sunday May 10, 2026 @07:56AM (#48083231)
It's close but not the same. Kernel PCA lifts all D coordinates which gives M around 525k at D = 1024. In the post I do PCA first to reduce D to d = 256, then lift only those d coordinates, M = 33k. Much smaller, much faster Ridge solve.
Re:A polynomial autoencoder beats PCA on transformer embeddings by ComplexSystems on Sunday May 10, 2026 @02:41PM (#48086568)
That makes sense. If you could magically just get the top d PCs in quadratic kernel space without having to compute the whole kernel matrix, and then just do top-d quadratic PCs -> ridge, would that be better than doing the PCA -> top-d -> quadratic kernel ridge as you are now?