News for hackers. Stuff that matters.
Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by minimaxir on Sunday May 18, 2025 @03:34PM (#44023791)
PCA having nice separation is extremely uncommon unless your data is unusually clean or has obvious patterns. Even for the comically-easy MNIST dataset, the PCA representation doesn't separate nicely: https://github.com/lmcinnes/umap_paper_notebooks/blob/master...
Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by jszymborski on Sunday May 18, 2025 @04:57PM (#44024267)
"extremely uncommon" is very much not my experience when dealing with well-trained embeddings.

I'd add that just because you can achieve separability from a method, the resulting visualization may not be super informative. The distance between clusters that appear in t-SNE projected space often have nothing to do with their distance in latent space, for example. So while you get nice separate clusters, it comes at the cost of the projected space greatly distorting/hiding the relationship between points across clusters.