News for hackers. Stuff that matters.
Re:A polynomial autoencoder beats PCA on transformer embeddings by pleshkov on Saturday May 09, 2026 @06:29PM (#48078897)
Just checked the normalization point. You were partially right, sqrt-normalization makes the difference x2 less. I'm updating the numbers in the post. Interesting moment. I did a smoke test of poly-AE without whitening, and the result didn't change. I won't mention it in the post cause right now I'm not sure if it's a random effect or really a polynomial lift compensates normalization

There are no comments on this comment yet.