News for hackers. Stuff that matters.
Re:Starting from scratch: Training a 30M Topological Transfo... by thesz on Sunday January 18, 2026 @05:56PM (#46673044)
For some reason people are still adding position encodings into embeddings.

As if they are not relying on the model's ability to develop its own "positional system bootstrapping on top of the barebones one."

There are no comments on this comment yet.