News for hackers. Stuff that matters.
Re:Inkling 2 Mini by montroser on Friday November 20, 2026 @03:40PM (#50581726)
This is bonkers good if these benchmarks hold up in real-world scenarios. To be at the level of Qwen3.8-27B with a looped transformer model is quite the flex. They just left this note on reddit:

Hope that folks find this model more amenable to local usage - the sizing was largely informed by what folks on this subreddit ask for the most haha!

There are two particularly interesting aspects of this model for local users:

We do our training end-to-end in nvfp4, and although we haven't exhaustively benchmarked this, folks like Unsloth have noticed that our models seem to respond very well to being quantized, maintaining 90% top-1 even at at 1-bit quantization.

This model is looped, which allows more depth at the same parameter count, which we particularly notice seems helpful for coding tasks. :)

This is our first model targeted at local users, so let us know how this model is and anything you want to see out of future iterations!

https://www.reddit.com/r/LocalLLaMA/comments/1y1diuk/thinkin...

Re:Inkling 2 Mini by LPisGood on Friday November 20, 2026 @06:09PM (#50583675)
This is the first I’m hearing of a looped model. What is it?
Re:Inkling 2 Mini by cmrdporcupine on Friday November 20, 2026 @06:12PM (#50583704)
Goes a couple or three times through the same weights in a middle layer of the model. Nemotron's Dory pioneered this I think. Seems to give better results with smaller models.

Not sure the impact on kvcache though.

Re:Inkling 2 Mini by teravor on Friday November 20, 2026 @06:39PM (#50583949)
I believe someone found out (DIY style) that looping a couple of layers (found through trial and error) in ordinary LLM's yielded some benefits quite a while ago. it was a surprise as there was no looping during training.

then there was a fully looped model (forget the name) where all the layers were looped (which is an obvious mistake as it's known LLM's have token processing and reasoning stages) and showed some decent results.

and now the concept worked itself into the mainstream.

Re:Inkling 2 Mini by calebkaiser on Friday November 20, 2026 @06:50PM (#50584055)
There's a lot of interesting work around this going back a little while. It's spiritually inline with the thinking around RNNs, which date back further. The Universal Transformer (2018) is the first research paper I'm familiar with that really pushed the idea as applied to transformer models, but I'm sure there are others (as is the case with everything in deep learning, someone tried it 8 years before you using different nomenclature).

If you're curious, it's a very cool paper! https://arxiv.org/html/1807.03819v1

Re:Inkling 2 Mini by dannyw on Friday November 20, 2026 @07:40PM (#50584466)
Yes, the DIY “LLM neurosurgery”: https://dnhkng.github.io/posts/rys/

The Show HN of someone applying this recently, through layer duplication: https://news.ycombinator.com/item?id=47431671

Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @06:30PM (#50583866)
Nvidia Dory is another (dense 9B) example.
Re:Inkling 2 Mini by kroaton on Friday November 20, 2026 @06:35PM (#50583907)
Supposedly GPT6 Astra uses this as well.
Re:Inkling 2 Mini by calebkaiser on Friday November 20, 2026 @06:46PM (#50584025)
It's a pretty broad bucket that contains a few different research lineages, but the gist (when talking about Transformers) is that instead of stacking 12 latent blocks, you can stack 4 and let them loop 3 times.

That's just a general simplification though. One of the key motivators in this general area of research is adaptive latent reasoning, meaning in theory, some tasks only require 2 loops and some require 8.

There's a lot of different approaches here and many of them approach the problem from different traditions. Some looped transformers use a fixed loop count, others follow after deep equilibrium models and iterate towards a fixed point or some other stopping mechanism.

Some fun reading if you're interested in this field:

- TinyRecursiveModels - Blew up a couple of years ago when it performed really well on ARC AGI 1 using a comparatively small model with a large task-level table of embedding prefixes. https://github.com/samsungsailmontreal/tinyrecursivemodels

- Sebastian Raschka's piece on the general architecture patterns we see today: https://sebastianraschka.com/llm-architecture-gallery/looped...

Re:Inkling 2 Mini by reissbaker on Friday November 20, 2026 @11:12PM (#50585862)
A couple other benefits aside from adaptive reasoning:

- It lets the model do more of its reasoning without emitting tokens. This is helpful both because a) insanely long token streams are annoying, but more importantly b) long reasoning streams eat context. It's not a direct performance enhancement, since each token is more expensive to generate (you need to loop!), but it reduces spam in the context window which likely translates into some degree of intelligence win for multi-turn since the model won't have to sift through as much noise, and it'll need to compact less frequently per task which is also likely an indirect intelligence win and an indirect cost/perf win per task: post-compaction re-reads of files are basically useless spam to an inference engine, so if you could do the entire task within one context window, you'd hit the engine less per task.

- It lets potentially every token use more computation, instead of keeping the test-time-compute scaling purely in the upfront CoT domain. While it's helpful to think upfront, it can be useful to think mid-stream too.

Overall it's potentially much better.

Re:Inkling 2 Mini by usef- on Friday November 20, 2026 @11:55PM (#50586066)
Yes, and for others' reference, Astra is the one SOTA model known to be looped.

Looping is controversial among the alignment/safey crowd, because the more models can think without printing a trace, the less understandable they might be to us.

Re:Inkling 2 Mini by stymaar on Friday November 20, 2026 @11:39PM (#50585999)
It's basically a generalization of the finding that repeating certain layers in an LLM makes it smarter: https://note.com/shi3zblog/n/nbb931d23e156?hl=en

But instead of duplicating the layer's weight, you run through them multiple times at runtime.

Re:Inkling 2 Mini by hashmap on Saturday November 21, 2026 @12:32AM (#50586262)
yeap, and 3x is where i saw the most gain from it too. you can set up most any model to do just this if you can find its reasoning layer boundaries, like qwen3.8 for example, h27-h35 will do it.
Re:Inkling 2 Mini by stavros on Friday November 20, 2026 @06:18PM (#50583759)
> This is bonkers good if these benchmarks hold up in real-world scenarios.

To me, this means they probably won't.

Re:Inkling 2 Mini by blurbleblurble on Friday November 20, 2026 @06:34PM (#50583902)
If not this model it's gonna be one of the whole slew on the way, especially once we start seeing looped diffusion (and similar) models. They're gonna smash your expectations.
Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @06:43PM (#50583998)
Every time I look at these I think, yes, another small model claims to outperform Muse Glimmer.

And then I test it in comparison and I remember Goodhart's Law.

This model, for sure, is interesting. But there's more than one way to peel a potato, and again and again I find that in practice these benchmarks don't really seem to mirror normal "coding assistant" uses.

Re:Inkling 2 Mini by ycui7 on Friday November 20, 2026 @07:49PM (#50584545)
why do people need to even quantize this model? it is already in nvfp4.
Re:Inkling 2 Mini by daemonologist on Friday November 20, 2026 @08:14PM (#50584758)
If your GPU has less than 24 GB of VRAM you might want it smaller. On non-Blackwell it probably doesn't make much speed difference. Also, requantizing to a more standard format might be needed for some engine/hardware combinations(?)
Re:Inkling 2 Mini by nico on Friday November 20, 2026 @09:05PM (#50585140)
> This is bonkers good if these benchmarks hold up in real-world scenarios

I think we’re going to start seeing a lot more of these types of advances, especially on smaller models

I’ve been able to graft reasoning and math problem solving into a small, locally trained model with 6.5M params. And I’ve also been able to add abilities or improve them by “growing” the mode to 8.5M params (this usually means redistributing the weights in a slightly different network size, then finetune the new size)

The model is so small that I can sometimes run dozens of experiments per day. There are a lot of papers and new models coming out. A lot of the stuff they are doing at 100B-1T still works at smaller scale (usually not the scaling inference-related optimizations, but rather things like weight programming, or a “looped” architecture). It’s a lot of fun to run at home on my laptop

I imagine bigger labs with bigger budgets (more GPUs) are mostly focused on 20B-1T+ models. But there’s a lot to explore in local “home models”

Re:Inkling 2 Mini by noodletheworld on Friday November 20, 2026 @11:42PM (#50586012)
People have been saying this since we first started having local models … that were too big to run locally.

It would be great if it was true.

Running a local frontier model is the dream.

…but, its been obvious since the beginning that this is a fantasy without a paradigm shift.

People want general purpose models. People want parallel models. People want fast models.

This is interesting, because it does represent a paradigm shift, perhaps… but most people saying “Ive been expecting this…” have simply not had a realistic understanding of complexities involved.

Personally; Im skeptical. This is what people were saying about 1 bit models. That turned out to make very little meaningful diff and did not result in a local hosting utopia.

I guess we will see… I feel like we will see slow incremental improvements in this space.

Sub 1B coding models remains a fantasy for now.

Re:Inkling 2 Mini by soerxpso on Saturday November 21, 2026 @12:27AM (#50586235)
When people started saying this, top-end frontier models were dumber than this model is. It's just that the frontier moves forward so fast that by the time the "paradigm shift" happens of small models being usable, there's a new batch of frontier models that open an entirely different paradigm that small models cannot contest.
Re:Inkling 2 Mini by nico on Saturday November 21, 2026 @12:33AM (#50586264)
> Sub 1B coding models remains a fantasy for now.

For sure, and it will continue to be so, until it isn’t

In the meantime, I think there’s a lot of encouraging developments

Re:Inkling 2 Mini by UncleOxidant on Saturday November 21, 2026 @12:38AM (#50586297)
I think that will remain the case for general purpose coding, however we might be able to train very focused coding models in that size. For example, a model that's only focused on coding Python - maybe that could fit in 1B params?
Re:Inkling 2 Mini by _factor on Friday November 20, 2026 @09:32PM (#50585319)
In my personal agentic benchmark, it’s doing even better.