Re:Understanding Positional Embeddings in Modern LLMs
by programjames on Sunday November 15, 2026 @03:29PM
(#50510379)
Most people who publish papers in ML are alchemists, but please do not assume all of us are. The original authors of the RoPE paper knowingly designed their embedding to be translation-invariant. I do not know if they were aware it was a group representation, but what they were trying to design was in essence a group representation.
[ Parent
| on HN ]
There are no comments on this comment yet.