Show HN: I modeled the Voynich Manuscript with SBERT to test for structure

Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by patcon on Sunday May 18, 2025 @02:49PM (#44023494)

I see that you're looking for clusters within PCA projections -- You should look for deeper structure with hot new dimensional reduction algorithms, like PaCMAP or LocalMAP!

I've been working on a project related to a sensemaking tool called Pol.is [1], but reprojecting its wiki survey data with these new algorithms instead of PCA, and it's amazing what new insight it uncovers with these new algorithms!

https://patcon.github.io/polislike-opinion-map-painting/

Painted groups: https://t.co/734qNlMdeh

(Sorry, only really works on desktop)

[1]: https://www.technologyreview.com/2025/04/15/1115125/a-small-...

[ Parent | on HN ]

Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by brig90 on Sunday May 18, 2025 @03:01PM (#44023585)

Thanks for pointing those out — I hadn’t seen PaCMAP or LocalMAP before, but that definitely looks like the kind of structure-preserving approach that would fit this data better than PCA. Appreciate the nudge — going to dig into those a bit more.

[ Parent | on HN ]

Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by staticautomatic on Sunday May 18, 2025 @04:03PM (#44023946)

I’ve had much better luck with umap than PCA and t-sne for reducing embeddings.

[ Parent | on HN ]

Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by patcon on Sunday May 18, 2025 @06:43PM (#44024851)

PaCMAP (and its descendant localmap) are comparable to t-sne at preserving both local and global structure (but without messing much with finicky hyperparameters)

https://youtu.be/sD-uDZ8zXkc

[ Parent | on HN ]

Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by loxias on Sunday May 18, 2025 @11:21PM (#44026285)

Try TDA ("mapper", or really, anything based on kernel density computed connectivity), it's a whole new world.

This ain't your parents' "factor analysis".

[ Parent | on HN ]

Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by patcon on Monday May 19, 2025 @12:24PM (#44031487)

Ooooo I will definitely check it out! It's strangely hard to find any comparisons in youtube videos -- it seems TDA isn't actually a dimensional reduction algorithm, but something closely relayed, maybe?

[ Parent | on HN ]

Re:Show HN: I modeled the Voynich Manuscript with SBERT to t... by khafra on Monday May 19, 2025 @07:04AM (#44028466)

LLM model interpretability also uses Sparse Autoencoders to find concept representations (https://openai.com/index/extracting-concepts-from-gpt-4/), and, more recently, linear probes.

[ Parent | on HN ]