Interesting, did you try also using PaCMAP or UMAP for dimensionality reduction? It might result in a more meaningful representation of their underlying semantic structure: see the 'mammoth' example in my article.