News for hackers. Stuff that matters.
Re:Inkling 2 Mini by lilytweed on Friday November 20, 2026 @04:43PM (#50582638)
Whoa, not just a model that acknowledges and embraces using a looped architecture (GPT Astra 6+ and Sol 6.1+ are only [heavily] rumored to be looped), but it's open weights, too?

I'm willing to wager that most open models will go looped in the wake of this.

Wonder what it means for safety and interpretability at this level of availability.

Re:Inkling 2 Mini by coder543 on Friday November 20, 2026 @04:55PM (#50582836)
[delayed]
Re:Inkling 2 Mini by twothreeone on Friday November 20, 2026 @06:20PM (#50583775)
I think that's really the key here right? It doesn't really "beat" Qwen3.8-27B.. more of like it's on par / a few points ahead on some, behind on others. But contrary to the Qwen (which is dense) it actually does it as an MoE - so less active parameters.

And the fact that they trained it as a looped model in 4-bit native is also kind of impressive to me as well, because it combines recurrence, routing, and narrow-format - all of which adversely impact numeric stability during training.

Re:Inkling 2 Mini by dannyw on Friday November 20, 2026 @07:54PM (#50584577)
Beyond that, they detailed quite a bit of their training pipeline for Inkling 2 Small.

Apparently trained on a ~couple dozen trillion tokens; which is far smaller than most pretraining and posttraining runs.

Re:Inkling 2 Mini by reissbaker on Friday November 20, 2026 @11:17PM (#50585892)
It beats Qwen on effectively every coding benchmark, which I think are the most relevant ones to HN :)

On the other benchmarks it trades blows or beats Qwen; as Thinky points out, looping helps reasoning but doesn't scale memorization, so tasks requiring broad world knowledge without tools/search won't be helped by looping.

Still super impressive, and probably replaces Qwen 27B in pretty much any workload!

Re:Inkling 2 Mini by stymaar on Friday November 20, 2026 @11:42PM (#50586011)
I wonder if one could build a looping model with PLE but with different embedding loaded at different iterations of the same layer.
Re:Inkling 2 Mini by XenophileJKO on Friday November 20, 2026 @05:09PM (#50583007)
This has been my line of thought for a years. If we are aiming for on device intelligence, looping (or other methods of parameter reuse) are one of the best methods since consumer hardware is primarily memory constrained.

Which is counter to the way most open weight models have been leaning. Many current models are MoE (Mixture of Experts) which by their very nature are parameter heavy. They are trading parameter efficiency for compute efficiency which is kind of the wrong tradeoff for current consumer hardware like gaming GPUs.

Re:Inkling 2 Mini by coder543 on Friday November 20, 2026 @05:18PM (#50583114)
[delayed]
Re:Inkling 2 Mini by Eisenstein on Friday November 20, 2026 @05:58PM (#50583556)
[delayed]
Re:Inkling 2 Mini by coder543 on Friday November 20, 2026 @06:04PM (#50583617)
[delayed]
Re:Inkling 2 Mini by stymaar on Friday November 20, 2026 @11:44PM (#50586018)
> since consumer hardware is primarily memory constrained.

It depends, people with Macs or AMD with unified memory are more constrained by memory bandwidth than with resident memory.

Unfortunately for us, looping doesn't help on that from.

Re:Inkling 2 Mini by cubefox on Friday November 20, 2026 @05:44PM (#50583404)
> Wonder what it means for safety and interpretability at this level of availability.

We already know from official model cards that the GPT-6 models are substantially less monitorable than other (non-looped) models. Which is not surprising, since they are doing some amount of Neuralese thinking between each token.