Re:A polynomial autoencoder beats PCA on transformer embeddings
by pleshkov on Saturday May 09, 2026 @06:29PM
(#48078897)
Just checked the normalization point. You were partially right, sqrt-normalization makes the difference x2 less. I'm updating the numbers in the post.
Interesting moment. I did a smoke test of poly-AE without whitening, and the result didn't change. I won't mention it in the post cause right now I'm not sure if it's a random effect or really a polynomial lift compensates normalization
[ Parent
| on HN ]
There are no comments on this comment yet.