Token Embeddings Violate the Manifold Hypothesis Posted by lukeplato on Monday November 10, 2025 @03:01PM from the arxiv.org dept. https://arxiv.org/abs/2504.01002 ( 1 comment | 2 points | on HN )
One thing discovered was that 800+ tokens converged to a single point in the 2500+ dimensional unembedding space
Some examples:
https://bsky.app/profile/jefferyharrell.bsky.social/post/3m4...
https://bsky.app/profile/jefferyharrell.bsky.social/post/3m5...