News for hackers. Stuff that matters.
Re:Inkling 2 Mini by calebkaiser on Friday November 20, 2026 @06:50PM (#50584055)
There's a lot of interesting work around this going back a little while. It's spiritually inline with the thinking around RNNs, which date back further. The Universal Transformer (2018) is the first research paper I'm familiar with that really pushed the idea as applied to transformer models, but I'm sure there are others (as is the case with everything in deep learning, someone tried it 8 years before you using different nomenclature).

If you're curious, it's a very cool paper! https://arxiv.org/html/1807.03819v1

There are no comments on this comment yet.