News for hackers. Stuff that matters.
Re:Inkling 2 Mini by cmrdporcupine on Friday November 20, 2026 @06:12PM (#50583704)
Goes a couple or three times through the same weights in a middle layer of the model. Nemotron's Dory pioneered this I think. Seems to give better results with smaller models.

Not sure the impact on kvcache though.

Re:Inkling 2 Mini by teravor on Friday November 20, 2026 @06:39PM (#50583949)
I believe someone found out (DIY style) that looping a couple of layers (found through trial and error) in ordinary LLM's yielded some benefits quite a while ago. it was a surprise as there was no looping during training.

then there was a fully looped model (forget the name) where all the layers were looped (which is an obvious mistake as it's known LLM's have token processing and reasoning stages) and showed some decent results.

and now the concept worked itself into the mainstream.

Re:Inkling 2 Mini by calebkaiser on Friday November 20, 2026 @06:50PM (#50584055)
There's a lot of interesting work around this going back a little while. It's spiritually inline with the thinking around RNNs, which date back further. The Universal Transformer (2018) is the first research paper I'm familiar with that really pushed the idea as applied to transformer models, but I'm sure there are others (as is the case with everything in deep learning, someone tried it 8 years before you using different nomenclature).

If you're curious, it's a very cool paper! https://arxiv.org/html/1807.03819v1

Re:Inkling 2 Mini by dannyw on Friday November 20, 2026 @07:40PM (#50584466)
Yes, the DIY “LLM neurosurgery”: https://dnhkng.github.io/posts/rys/

The Show HN of someone applying this recently, through layer duplication: https://news.ycombinator.com/item?id=47431671