News for hackers. Stuff that matters.
Re:Inkling 2 Mini by mattnewton on Friday November 20, 2026 @03:21PM (#50581428)
Benchmarks look good, and inkling 2 small felt better than the benchmarks suggested, so I am very excited to play with this.

From a research perspective, I'm even more interested in the varlooped variant they're releasing https://thinkingmachines.ai/model-card/tml-d24-varloop-resea... I'm not aware of any other working open source models like that.

Re:Inkling 2 Mini by kcb on Friday November 20, 2026 @03:30PM (#50581565)
Re:Inkling 2 Mini by liuliu on Friday November 20, 2026 @03:35PM (#50581652)
My current feedback on Inkling 2 Small: 1. it feels intelligent, and can solve open problems throw at it like other quasi-frontier models (DeepSeek 4 Flash 0731 / DeepSeek 4.1 Flash 1017 / GLM 5.3 Flash etc). 2. its responses sometimes can be odd (they are honest about it in the blog post!), can have wrong use of markdown syntax, sometimes overly brief, etc. 3. tool uses are good, but sometimes have some old take on tools (too quickly to assume image generation tools cannot do text etc).

EDIT: by "open problems", I meant the problems I don't really formulate how to solve it first in my mind and then throw to it. I just describe what I want and let it to figure it out.

Re:Inkling 2 Mini by walrus01 on Friday November 20, 2026 @03:27PM (#50581530)
Looking at the link to quantizations it looks like unsloth has already quantized it:

https://huggingface.co/thinkingmachines/Inkling-2-Mini

https://huggingface.co/unsloth/Inkling-2-Mini-GGUF

In terms of the active parameters and token/s on slow or less capable hardware it seems like it should be in the same general performance category as qwen 3.6 35B A3B.

Re:Inkling 2 Mini by jakswa on Friday November 20, 2026 @03:47PM (#50581829)
might need this if trying to test on llama.cpp https://github.com/ggml-org/llama.cpp/pull/32695 -- "Adds support for https://huggingface.co/thinkingmachines/Inkling-2-Mini - uses a looped architecture and this PR adds looped support"
Re:Inkling 2 Mini by jakswa on Friday November 20, 2026 @04:00PM (#50582015)
Re:Inkling 2 Mini by hrncn on Friday November 20, 2026 @04:13PM (#50582199)
Let's hope these make it into master, and don't end up like the diffusiongemma patchsets. It is always a bit sad if the exotic and exciting models don't make it into master and all the people using prebuilt llama can't just give them a try. Tho I can understand why these in particular might not be wanted in master.
Re:Inkling 2 Mini by neya on Friday November 20, 2026 @03:32PM (#50581587)
Wow, that coding demo was super impressive. Can't wait to try it out. Just need to wait for the hardware market to cool down a bit so we can run these on our own commodity hardware.
Re:Inkling 2 Mini by christkv on Friday November 20, 2026 @03:33PM (#50581614)
I'm loving this trend. My framework desktop is looking like a better and better investment.
Re:Inkling 2 Mini by topaztee on Friday November 20, 2026 @03:56PM (#50581949)
tell me more about this framework desktop..
Re:Inkling 2 Mini by christkv on Friday November 20, 2026 @04:50PM (#50582754)
It's just a ryzen 395 ai max box with 128GB but that I was lucky to buy before the ramagedon happened. It just happens to run a fair bit of open source models decently and the models are getting better. I got it for 2100 EUR and now it looks like a steal.
Re:Inkling 2 Mini by joerambo808 on Friday November 20, 2026 @06:53PM (#50584099)
I left a framework desktop in my cart and came back after the price increase. You definitely got a steal.
Re:Inkling 2 Mini by rapind on Friday November 20, 2026 @04:00PM (#50582021)
What are you running on it? DSF 4.1?
Re:Inkling 2 Mini by christkv on Friday November 20, 2026 @04:48PM (#50582723)
DSF 4.1 and Qwen 3.8 27b. I have to switch linux distro though been on Bazzite and the Fedora Atomic underpinnings is just killing me for this stuff just makes it harder than it needs to be to install stuff.
Re:Inkling 2 Mini by rsolva on Friday November 20, 2026 @06:08PM (#50583663)
Just do it all in podman containers! I run stock Fedora Silverblue and do every project in containers these days, keeps the OS clean and tidy.
Re:Inkling 2 Mini by Iolaum on Friday November 20, 2026 @04:02PM (#50582061)
Indeed. I 've been using Sage A1.5 35b-A3b to have more speed because of strix halo limited memory bandwidth. Looking forward to testing Inkling-2-mini once it has lamma.cpp support. Genuinely excited from a model outside Qwen3.6-35B-A3B ecosystem competing in that space.

P.S. Qwen3.8-Flash-Next is also a very good model for Srix-Halo/128Gb Framework Desktop but I liked to have more RAM for other things as well even when I was running it in Q4. Usable though.

Re:Inkling 2 Mini by montroser on Friday November 20, 2026 @03:40PM (#50581726)
This is bonkers good if these benchmarks hold up in real-world scenarios. To be at the level of Qwen3.8-27B with a looped transformer model is quite the flex. They just left this note on reddit:

Hope that folks find this model more amenable to local usage - the sizing was largely informed by what folks on this subreddit ask for the most haha!

There are two particularly interesting aspects of this model for local users:

We do our training end-to-end in nvfp4, and although we haven't exhaustively benchmarked this, folks like Unsloth have noticed that our models seem to respond very well to being quantized, maintaining 90% top-1 even at at 1-bit quantization.

This model is looped, which allows more depth at the same parameter count, which we particularly notice seems helpful for coding tasks. :)

This is our first model targeted at local users, so let us know how this model is and anything you want to see out of future iterations!

https://www.reddit.com/r/LocalLLaMA/comments/1y1diuk/thinkin...

Re:Inkling 2 Mini by LPisGood on Friday November 20, 2026 @06:09PM (#50583675)
This is the first I’m hearing of a looped model. What is it?
Re:Inkling 2 Mini by cmrdporcupine on Friday November 20, 2026 @06:12PM (#50583704)
Goes a couple or three times through the same weights in a middle layer of the model. Nemotron's Dory pioneered this I think. Seems to give better results with smaller models.

Not sure the impact on kvcache though.

Re:Inkling 2 Mini by teravor on Friday November 20, 2026 @06:39PM (#50583949)
I believe someone found out (DIY style) that looping a couple of layers (found through trial and error) in ordinary LLM's yielded some benefits quite a while ago. it was a surprise as there was no looping during training.

then there was a fully looped model (forget the name) where all the layers were looped (which is an obvious mistake as it's known LLM's have token processing and reasoning stages) and showed some decent results.

and now the concept worked itself into the mainstream.

Re:Inkling 2 Mini by calebkaiser on Friday November 20, 2026 @06:50PM (#50584055)
There's a lot of interesting work around this going back a little while. It's spiritually inline with the thinking around RNNs, which date back further. The Universal Transformer (2018) is the first research paper I'm familiar with that really pushed the idea as applied to transformer models, but I'm sure there are others (as is the case with everything in deep learning, someone tried it 8 years before you using different nomenclature).

If you're curious, it's a very cool paper! https://arxiv.org/html/1807.03819v1

Re:Inkling 2 Mini by dannyw on Friday November 20, 2026 @07:40PM (#50584466)
Yes, the DIY “LLM neurosurgery”: https://dnhkng.github.io/posts/rys/

The Show HN of someone applying this recently, through layer duplication: https://news.ycombinator.com/item?id=47431671

Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @06:30PM (#50583866)
Nvidia Dory is another (dense 9B) example.
Re:Inkling 2 Mini by kroaton on Friday November 20, 2026 @06:35PM (#50583907)
Supposedly GPT6 Astra uses this as well.
Re:Inkling 2 Mini by calebkaiser on Friday November 20, 2026 @06:46PM (#50584025)
It's a pretty broad bucket that contains a few different research lineages, but the gist (when talking about Transformers) is that instead of stacking 12 latent blocks, you can stack 4 and let them loop 3 times.

That's just a general simplification though. One of the key motivators in this general area of research is adaptive latent reasoning, meaning in theory, some tasks only require 2 loops and some require 8.

There's a lot of different approaches here and many of them approach the problem from different traditions. Some looped transformers use a fixed loop count, others follow after deep equilibrium models and iterate towards a fixed point or some other stopping mechanism.

Some fun reading if you're interested in this field:

- TinyRecursiveModels - Blew up a couple of years ago when it performed really well on ARC AGI 1 using a comparatively small model with a large task-level table of embedding prefixes. https://github.com/samsungsailmontreal/tinyrecursivemodels

- Sebastian Raschka's piece on the general architecture patterns we see today: https://sebastianraschka.com/llm-architecture-gallery/looped...

Re:Inkling 2 Mini by reissbaker on Friday November 20, 2026 @11:12PM (#50585862)
A couple other benefits aside from adaptive reasoning:

- It lets the model do more of its reasoning without emitting tokens. This is helpful both because a) insanely long token streams are annoying, but more importantly b) long reasoning streams eat context. It's not a direct performance enhancement, since each token is more expensive to generate (you need to loop!), but it reduces spam in the context window which likely translates into some degree of intelligence win for multi-turn since the model won't have to sift through as much noise, and it'll need to compact less frequently per task which is also likely an indirect intelligence win and an indirect cost/perf win per task: post-compaction re-reads of files are basically useless spam to an inference engine, so if you could do the entire task within one context window, you'd hit the engine less per task.

- It lets potentially every token use more computation, instead of keeping the test-time-compute scaling purely in the upfront CoT domain. While it's helpful to think upfront, it can be useful to think mid-stream too.

Overall it's potentially much better.

Re:Inkling 2 Mini by stavros on Friday November 20, 2026 @06:18PM (#50583759)
> This is bonkers good if these benchmarks hold up in real-world scenarios.

To me, this means they probably won't.

Re:Inkling 2 Mini by blurbleblurble on Friday November 20, 2026 @06:34PM (#50583902)
If not this model it's gonna be one of the whole slew on the way, especially once we start seeing looped diffusion (and similar) models. They're gonna smash your expectations.
Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @06:43PM (#50583998)
Every time I look at these I think, yes, another small model claims to outperform Muse Glimmer.

And then I test it in comparison and I remember Goodhart's Law.

This model, for sure, is interesting. But there's more than one way to peel a potato, and again and again I find that in practice these benchmarks don't really seem to mirror normal "coding assistant" uses.

Re:Inkling 2 Mini by ycui7 on Friday November 20, 2026 @07:49PM (#50584545)
why do people need to even quantize this model? it is already in nvfp4.
Re:Inkling 2 Mini by daemonologist on Friday November 20, 2026 @08:14PM (#50584758)
If your GPU has less than 24 GB of VRAM you might want it smaller. On non-Blackwell it probably doesn't make much speed difference. Also, requantizing to a more standard format might be needed for some engine/hardware combinations(?)
Re:Inkling 2 Mini by nico on Friday November 20, 2026 @09:05PM (#50585140)
> This is bonkers good if these benchmarks hold up in real-world scenarios

I think we’re going to start seeing a lot more of these types of advances, especially on smaller models

I’ve been able to graft reasoning and math problem solving into a small, locally trained model with 6.5M params. And I’ve also been able to add abilities or improve them by “growing” the mode to 8.5M params (this usually means redistributing the weights in a slightly different network size, then finetune the new size)

The model is so small that I can sometimes run dozens of experiments per day. There are a lot of papers and new models coming out. A lot of the stuff they are doing at 100B-1T still works at smaller scale (usually not the scaling inference-related optimizations, but rather things like weight programming, or a “looped” architecture). It’s a lot of fun to run at home on my laptop

I imagine bigger labs with bigger budgets (more GPUs) are mostly focused on 20B-1T+ models. But there’s a lot to explore in local “home models”

Re:Inkling 2 Mini by _factor on Friday November 20, 2026 @09:32PM (#50585319)
In my personal agentic benchmark, it’s doing even better.
Re:Inkling 2 Mini by jakswa on Friday November 20, 2026 @03:41PM (#50581731)
The masses lying in wait for Qwen 4 cry out in joy. A savior? Could it be?!
Re:Inkling 2 Mini by jakswa on Friday November 20, 2026 @03:44PM (#50581788)
When inkling first released I tested it out and thought it was the best prose from an LLM in my quick sampling -- easy-reading and easy to understand. Big hopes I can write code with one of these models one day.
Re:Inkling 2 Mini by summarity on Friday November 20, 2026 @03:51PM (#50581882)
Absolutely. Inkling for prose (and vision), Deepseek 10XX for code are the sweet spot right now (they swap places for these jobs on my rig)
Re:Inkling 2 Mini by holistio on Friday November 20, 2026 @03:46PM (#50581810)
I'm starting to believe Gestalt is actually theirs. This is very nice.
Re:Inkling 2 Mini by efficient_dairy on Friday November 20, 2026 @06:45PM (#50584016)
No. It's either some router that sends the query to the best possible model or just a Chinese (possibly bench-maxed) model. US models are never stealth.
Re:Inkling 2 Mini by mkl on Friday November 20, 2026 @07:26PM (#50584372)
> US models are never stealth.

Not true. Google's Nano Banana, OpenAI's Quasar Alpha and Optimus Alpha [0], Horizon Alpha and Beta [1].

[0] https://openrouter.ai/blog/announcements/quasar-alpha-and-op...

[1] https://openrouter.ai/blog/announcements/gpt-5-is-now-live/

Re:Inkling 2 Mini by gchadwick on Friday November 20, 2026 @03:47PM (#50581821)
Very interesting to see them publish a looped transformer. It's been reported that OpenAI are now using looped transformers, though of course not confirmed.

I think this may be the first public example of a looped transformer model good enough for real world use cases?

Re:Inkling 2 Mini by rsolva on Friday November 20, 2026 @03:58PM (#50581985)
Going to test this right away on my AMD 7900XTX and see how it does compared to Qwen 3.8 27B (or DaisyRose 27B that I'm currently running). Given that Inkling 2 Mini is a MoE, it should provide way higher decode speed, but with comparable performance. Exiting!
Re:Inkling 2 Mini by AndrewKemendo on Friday November 20, 2026 @04:18PM (#50582266)
Please reply when you have tested it I’m curious.

Also do you know what the best local setup is?

Re:Inkling 2 Mini by rsolva on Friday November 20, 2026 @04:27PM (#50582419)
For 7900XTX you mean? I run Fedora Silverblue 44 at the moment and could share my current containerized setup with DaisyRose 27B.
Re:Inkling 2 Mini by jakswa on Friday November 20, 2026 @04:30PM (#50582454)
I had astra patch in both required llama.cpp PRs and it's passing a few tests locally on my 7900XTX so far, but ~500 tok/s prefill and ~26 tok/s decode are sobering: The new three-time-looping might be cutting speeds. Maybe there's something here still though, maybe it needs less reasoning output for results to be usable/good? More testing needed!

edit: found some oudated mesa/radv/shaderc/glslang packages to update, and up to 34 tok/s now. MORE MORE

Re:Inkling 2 Mini by rsolva on Friday November 20, 2026 @04:35PM (#50582531)
Hm, that seems strangely slow. Qwen3.8-27B/DaisyRose-27B Q4 currently run with ~66 t/s and ~800 t/s prefill with 3 slots and about 190k kv cache. It performs really well so far.
Re:Inkling 2 Mini by jakswa on Friday November 20, 2026 @04:37PM (#50582563)
Yeah I've had 27B (dense model) up to similar speeds on this setup as well. I'll hold off to compare my speeds to others, maybe it's too early or it's particular to my setup.
Re:Inkling 2 Mini by enlyth on Friday November 20, 2026 @04:53PM (#50582795)
I'm getting 700 tok/s prefill and 52tok/s generation on a 5090 RTX, not bad
Re:Inkling 2 Mini by jakswa on Friday November 20, 2026 @05:08PM (#50582994)
5090 RTX is like 1800GB/s? I bet you were seeing much higher speeds on Qwen 3.8 27B right?
Re:Inkling 2 Mini by shadowpho on Friday November 20, 2026 @05:25PM (#50583203)
That seems slow, should be much much faster hmmm
Re:Inkling 2 Mini by baoooooooooooo on Friday November 20, 2026 @10:15PM (#50585549)
I'm always on the lookout for reports from my 3090 friends

Looking a bit oof on reddit rn, I'm seeing reports of 10 tok/s via llama.cpp :|

https://www.reddit.com/r/LocalLLaMA/comments/1y1g9cr/inkling...

Edit: Hopeful that this gets the ninfer treatment, if the benchmarks are to be believed it feels like there is probably both amazing speed and real performance to be unlocked by the community here on the x090 cards

Re:Inkling 2 Mini by IAmGraydon on Friday November 20, 2026 @04:51PM (#50582769)
>Exiting!

OK, but please come back and let us know your results!

Re:Inkling 2 Mini by rsolva on Friday November 20, 2026 @05:06PM (#50582966)
Got it to run, but a few bumps in the road – mostly because this is a very fresh model.

Flash Attention (FA) is not supported on the current llama.cpp (at least not for ROCm) for this model so KV cache is forced to f16 for the value (V) and I cannot lower it to q4 go get decent KV cache. But the key (K) could be set to q4, and that got me about 100k of ctx. Current benching shows ~28 tok/s decode and ~481 tok/s prefill.

It's a start, but I expected better speeds even on a non ideal setup. I'll keep experimenting!

Re:Inkling 2 Mini by xlayn on Friday November 20, 2026 @04:55PM (#50582827)
Hey, just in case.. I have been maintaining this fork of llama.cpp, I run also 7900XTX, 3 in my case... and I want to know if you see improved speed... notice the ngram mod where instead of loosing the ngrams they are stored in disk, I have a 31M slot and so far it's being working really nice for me

I'm seing around 2800t/s on pp at the very beginning of the conversation, and then going to around 800 by the 256k context size...

I've also update the value in the gguf to have instead of context size 256k be triple and have use it all the way to around 600k context size

use the cache eviction branch https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev...

llama-server -m /path_to_model/Qwen3.8-27B-UD-IQ4XS_u.gguf -np 2 --host 0.0.0.0 --port 8089 --kv-unified --temp 0.2 --top-k 20 --min-p 0.0 --cache-disk-path /path_to_store_conversations_to_disk --cache-ram 100 --cpu-range 0-23 --reasoning-budget 4096 --reasoning-budget-message "... I am thinking for too long -- let me gather more info about the task." --no-cache-idle-slots --spec-draft-auto --spec-draft-sampling probabilistic --spec-draft-n-min 1 --spec-draft-p-min 0 --spec-ngram-auto --spec-ngram-mod-size 35544320 --spec-ngram-mod-cache /path_to_storage_of_ngram_mod_this_is_specific_per_model/ngram-mod-qwen3.8-27b.bin --fit on --device Vulkan3,Vulkan2,Vulkan1 --fit-target 1024,2560,2560 --ctx-checkpoints 25 --checkpoint-spill-max 25 --cache-type-k f16 --cache-type-v f16 -ctkd q8_0 -ctvd q8_0 -devd Vulkan3 --spec-type draft-mtp,ngram-mod --spec-draft-n-max 7 --spec-ngram-mod-n-min 7 --spec-ngram-draft-n-max 31 --spec-rs-depth 24 --fit-verify on

Use this if you have 3 cards, if you have less maybe just remove it -b 8192

In case you want to know how many tokens you have use, --token-ledger /path_to_where_you_want_to_store_stats/token_ledger.json

here is how logs look like I saw a 350t/s on tg but was because you hit ngram mod tables over things the model has already seen

457.04.985.808 I srv print_line: token ledger: requests = 27077, prompt = 3928887769 (cached = 3874707808, processed = 47151296), generated = 22126308 (reasoning = 15129037), t> 457.04.985.913 I slot rq_write_fin: id 0 | task -1 | request timeline: task 77431, ttft 1580.0 ms (http 20.4, tokenize 65.1, queue 247.1, select 0.0, restore 2.7, prompt 1244.3),> 457.05.202.640 I srv handle_compl: task created: conversation_id=64d89c6d7ff76923 (tokens 0..18489, system+first_user) 457.05.202.757 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.994 (> 0.100 thold), f_keep = 1.000 457.05.206.552 I slot launch_slot_: id 0 | task 77458 | processing task, is_child = 0 457.05.207.188 I slot operator(): id 0 | task 77458 | prompt processing batch, tokens 81032..81036 of 81552 457.05.246.405 I slot operator(): id 0 | task 77458 | prompt processing batch, tokens 81036..81548 of 81552 457.05.310.149 I srv lazify_check: checkpoint: converted pos_min=80943 to disk-backed ('cp_64d89c6d7ff76923_80943.bin', 149.6 MiB released) 457.05.872.722 I slot operator(): id 0 | task 77458 | prompt processing batch, tokens 81548..81552 of 81552 457.05.943.921 I srv lazify_check: checkpoint: converted pos_min=81035 to disk-backed ('cp_64d89c6d7ff76923_81035.bin', 149.6 MiB released) 457.09.000.990 I slot print_timing: id 0 | task 77458 | n_gen = 340, tg = 112.88 t/s, tg_3s = 113.20 t/s 457.11.161.223 I slot print_timing: id 0 | task 77458 | prompt eval time = 790.96 ms / 520 tokens ( 1.52 ms per token, 657.43 tokens per second) 457.11.161.226 I slot print_timing: id 0 | task 77458 | eval time = 5163.27 ms / 642 tokens ( 8.06 ms per token, 124.15 tokens per second) 457.11.161.226 I slot print_timing: id 0 | task 77458 | total time = 5954.23 ms / 1162 tokens 457.11.161.227 I slot print_timing: id 0 | task 77458 | graphs reused = 72319 (pp skipped 525) 457.11.161.228 I slot print_timing: id 0 | task 77458 | graphs prebuilt = 7295 (hit 6975, miss 320) 457.11.161.229 I slot print_timing: id 0 | task 77458 | early prebuilt = 28686 (fit 44246, hit 26411, miss 2275, aborted 0, 96584.2 ms host) 457.11.161.238 I slot print_timing: id 0 | task 77458 | draft acceptance = 0.83204 ( 535 accepted / 643 generated), mean len = 6.00 457.11.161.243 I slot print_timing: id 0 | task 77458 | draft auto = n=6 | 1:55.4 t/s [2.00 tok / 36.1 ms] (4) 2:67.7 t/s [2.50 tok / 37.0 ms] (4) 3:101.0 t/s [4.00 tok / 39.6 ms> 457.11.161.251 I slot print_timing: id 0 | task 77458 | draft by impl: draft-mtp | steps 101 | built 5.5 | offered 5.5 | accepted 4.9 (0.90) | full 80 | step 45.2 ms | 131.5 t/s > 457.11.161.257 I slot print_timing: id 0 | task 77458 | draft by impl: ngram-mod | steps 6 | built 64.0 | offered 14.3 | accepted 6.0 (0.42) | full 1 | step 90.8 ms | 77.1 t/s | > 457.11.161.258 I slot print_timing: id 0 | task 77458 | ngram policy: auto on | cap now 12 | cap min 7 median 12 max 31 over 6 chains | cap bound 6 | whole ema 0.17 | resume 0 | > 457.11.161.259 I slot print_timing: id 0 | task 77458 | spec rollback: prebatch restores 1 | host checkpoints 0 | checkpoint restores 0 457.11.161.260 I slot print_timing: id 0 | task 77458 | chain truth: ngram | chains 6 | censored 0 | would land mean 7.0 | full 0 | hist 0:0 1-4:1 5-8:3 9-16:2 17-32:0 33-48:0 49> 457.11.161.261 I slot print_timing: id 0 | task 77458 | ngram gate: held 38 | agree 5 (would land 0:0 1-4:1 5-16:4 17+:0, pending 0) | disagree 33 (would land 0:33 1-4:0 5-16:0 1> 457.11.161.261 I slot print_timing: id 0 | task 77458 | ext shadow: mtp+ngram | attempts 101 | hits 85 | tested 68 | landed mean 0.4 (over tested) | hist 0:65 1-4:1 5-8:0 9-16:2 > 457.11.161.262 I slot print_timing: id 0 | task 77458 | alt table: n=48 | attempts 101 | hits 45 | would land mean 0.0 | hist 0:45 1-4:0 5-8:0 9-16:0 17-32:0 33-48:0 49-64:0 65+:> 457.11.161.263 I slot print_timing: id 0 | task 77458 | mtp ahead: tried 53 (0 after a p_min stop) | verify full 49 | bonus matched 48 (0 after p_min) | too short 0 | served 48 s> 457.11.161.265 I slot print_timing: id 0 | task 77458 | mtp confidence: stop cap:101(acc 4.9) pmin:0(acc 0.0) other:0 | p bins landed/tested: <0.5 1/3 0.5 0/1 0.6 1/1 0.7 0/0 0.8> 457.11.163.234 I slot release: id 0 | task 77458 | stop processing: n_tokens = 82195, truncated = 0 457.11.163.426 I srv print_line: token ledger: requests = 27078, prompt = 3928969321 (cached = 3874788840, processed = 47151816), generated = 22126950 (reasoning = 15129071), t> 457.11.163.526 I slot rq_write_fin: id 0 | task -1 | request timeline: task 77458, ttft 884.3 ms (http 21.2, tokenize 68.3, queue 0.1, select 0.0, restore 3.3, prompt 791.0), gen> 457.11.396.231 I srv handle_compl: task created: conversation_id=64d89c6d7ff76923 (tokens 0..18489, system+first_user)

Re:Inkling 2 Mini by yread on Friday November 20, 2026 @04:01PM (#50582031)
Recently SensaNova also released a looping model with unbelievable numbers

https://www.sensetime.com/en/news/sensenova-loop-9b-release-...

Re:Inkling 2 Mini by lwarfield on Friday November 20, 2026 @04:11PM (#50582173)
Omg I've been working on the same thing! I was doing this off a hunch from David Noel ng's RYS posts!

https://dnhkng.github.io/posts/rys/

Re:Inkling 2 Mini by mapontosevenths on Friday November 20, 2026 @05:11PM (#50583036)
I tried that for awhile too. Fun tip: You don't have to retrain the entire model after looping it to make it coherent again.. Just pick layers carefully then repair the residual by doing some parent distillation. ;)

That took me from needing to rent space in avast to just running it locally on my sparks and getting comparable results.

I also discovered that you can fix a lot of the damage caused by quantization the same way. Just do 200-500 rounds of parent student distillation with everything frozen but the residual, often the quantized version will improve drastically.

Re:Inkling 2 Mini by AndrewKemendo on Friday November 20, 2026 @04:17PM (#50582255)
Can anyone running one of these locally give any sense of how good they are?
Re:Inkling 2 Mini by 2001zhaozhao on Friday November 20, 2026 @04:18PM (#50582284)
I think this model will be a crazy big deal, it might be the first model that can reliably run multi-agent reasoning harnesses on local hardware, as this is a CoT model and can have several agents each with their own KV caches that fit into VRAM.

Dethroning Qwen 3.8 is a big deal too, and I like their section about CoT monitorability. I guess TML really are a bunch of rationalists who know what they're doing, sort of like Anthropic.

Re:Inkling 2 Mini by bpavuk on Friday November 20, 2026 @04:56PM (#50582850)
> TML really are a bunch of rationalists who know what they're doing

yes

> sort of like Anthropic.

no. Anthropic are not "rationalists," they are absolute fear-mongers. from "no software developers in 6 months" to "this could enable bioterrorism so we do not even let you mention the word 'virus' in any context", I do not see a hint of rationality.

Re:Inkling 2 Mini by ungovernableCat on Friday November 20, 2026 @05:14PM (#50583069)
I wonder how Ant's corporate culture will change now that so many employees will have secured some quite luxurious nest eggs.

Maybe they'll chill out a bit?

Re:Inkling 2 Mini by thereader12 on Friday November 20, 2026 @05:14PM (#50583077)
I mean I thinkhe meant like big R Rationalist like Effective Altruism or whatever other spin offs is du jour.
Re:Inkling 2 Mini by xvector on Friday November 20, 2026 @06:59PM (#50584156)
I'm gonna bet you'd be complaining a lot more if a bioterrorist used Claude and their blocks weren't strict enough.
Re:Inkling 2 Mini by sanex on Friday November 20, 2026 @07:32PM (#50584422)
I think you underestimate the actual difficulty of engaging in bioterrorism even if you give people access to the knowledge required. Listen to the story of the guy who treated his dog's cancer in Australia. It took a ton of access to university labs and scientists plus some luck. It's not like just mixing together some shit from the grocery store and if someone is able to accomplish that I would seriously discount Claudes contribution. Much like the mathematical proofs coming out these are not normies publishing them.
Re:Inkling 2 Mini by avazhi on Friday November 20, 2026 @07:34PM (#50584436)
You don’t know what he been by Rationalists, clearly.
Re:Inkling 2 Mini by lenerdenator on Friday November 20, 2026 @04:20PM (#50582311)
This looks like it could run on my M2 32GB MBP.

I honestly would rather have nice, focused-purpose local models than huge "let's make AGI" datacenter-hosted models. Will have to try it tonight when I get home.

Re:Inkling 2 Mini by segmondy on Friday November 20, 2026 @05:21PM (#50583145)
32gb is more than enough to run it!
Re:Inkling 2 Mini by gok on Friday November 20, 2026 @04:22PM (#50582337)
[delayed]
Re:Inkling 2 Mini by xlayn on Friday November 20, 2026 @04:33PM (#50582497)
When you are sure enough about what you are doing to include Qwen 3.8 27B in your compare.. we know you are talking... let's test drive it!
Re:Inkling 2 Mini by monksy on Friday November 20, 2026 @04:35PM (#50582543)
The GGUF version of this that can run on Ollama/Llama.cpp is: https://huggingface.co/unsloth/Inkling-2-Mini-GGUF
Re:Inkling 2 Mini by WiSaGaN on Friday November 20, 2026 @04:37PM (#50582571)
This is actually a big deal. I think probably by 2028 we will get a large number of daily tasks done reliably by laptop size local models.
Re:Inkling 2 Mini by stillpointlab on Friday November 20, 2026 @04:40PM (#50582604)
I'm guessing one of these local model companies will get picked by Apple sooner or later. It is just so within their business model and they have the hardware.
Re:Inkling 2 Mini by halJordan on Friday November 20, 2026 @04:51PM (#50582774)
Apple refuses to pay ai engineers an ai engineer salary. They're not going to acquire one of these companies until the cost of issuing them is within their comfort zone.
Re:Inkling 2 Mini by KeplerBoy on Friday November 20, 2026 @05:04PM (#50582949)
Do you really think ai engineers will forever be able to command 2026 ai engineer salaries?
Re:Inkling 2 Mini by bigyabai on Friday November 20, 2026 @04:55PM (#50582821)
Probably not for the foreseeable future. Apple penned a "multi-year collaboration" with Google to build their foundation models for them: https://blog.google/company-news/inside-google/company-annou...
Re:Inkling 2 Mini by kypro on Friday November 20, 2026 @04:43PM (#50582637)
> Efficient intelligence makes Inkling 2 Mini a great choice for inference where speed is paramount, or for running on local hardware.

Presumably v. good for offensive use cases too.

And a looped model?

This and GRH is +0.1% on my p-doom today.

The trends of capabilities and efficiency we've seen over the last few weeks should terrify anyone with any sense.

This will not end well but we'll have some nice AI generated dashboards until then I guess.

Re:Inkling 2 Mini by bigyabai on Friday November 20, 2026 @05:04PM (#50582942)
Anyone following the news is aware that we've had Mythos-comparable cyber red-teaming capabilities for quite a few months now. The sky hasn't fallen, instead people have been using it for reverse engineering, postmortems, pentesting and decomp projects. In a world where frontier labs and governments can coordinate large-scale cyberattacks with AI, having an uncensored open cyber model is necessary for defending against them. The alternative is much worse.

But hey, you're welcome to increment your P(doom) until the year 2057 if it makes you feel better. Maybe the government will finally put their foot down with the release of Qwen 28, which according to OpenAnthropicTimeWarnerPepsico will have unprecedented puppy-kicking use cases that necessitate regulation.

Re:Inkling 2 Mini by hypfer on Friday November 20, 2026 @05:13PM (#50583057)
May I suggest adding Comcast to that name
Re:Inkling 2 Mini by bigyabai on Friday November 20, 2026 @05:21PM (#50583158)
Not without triggering antitrust concerns, no.
Re:Inkling 2 Mini by lilytweed on Friday November 20, 2026 @04:43PM (#50582638)
Whoa, not just a model that acknowledges and embraces using a looped architecture (GPT Astra 6+ and Sol 6.1+ are only [heavily] rumored to be looped), but it's open weights, too?

I'm willing to wager that most open models will go looped in the wake of this.

Wonder what it means for safety and interpretability at this level of availability.

Re:Inkling 2 Mini by coder543 on Friday November 20, 2026 @04:55PM (#50582836)
[delayed]
Re:Inkling 2 Mini by twothreeone on Friday November 20, 2026 @06:20PM (#50583775)
I think that's really the key here right? It doesn't really "beat" Qwen3.8-27B.. more of like it's on par / a few points ahead on some, behind on others. But contrary to the Qwen (which is dense) it actually does it as an MoE - so less active parameters.

And the fact that they trained it as a looped model in 4-bit native is also kind of impressive to me as well, because it combines recurrence, routing, and narrow-format - all of which adversely impact numeric stability during training.

Re:Inkling 2 Mini by dannyw on Friday November 20, 2026 @07:54PM (#50584577)
Beyond that, they detailed quite a bit of their training pipeline for Inkling 2 Small.

Apparently trained on a ~couple dozen trillion tokens; which is far smaller than most pretraining and posttraining runs.

Re:Inkling 2 Mini by reissbaker on Friday November 20, 2026 @11:17PM (#50585892)
It beats Qwen on effectively every coding benchmark, which I think are the most relevant ones to HN :)

On the other benchmarks it trades blows or beats Qwen; as Thinky points out, looping helps reasoning but doesn't scale memorization, so tasks requiring broad world knowledge without tools/search won't be helped by looping.

Still super impressive, and probably replaces Qwen 27B in pretty much any workload!

Re:Inkling 2 Mini by XenophileJKO on Friday November 20, 2026 @05:09PM (#50583007)
This has been my line of thought for a years. If we are aiming for on device intelligence, looping (or other methods of parameter reuse) are one of the best methods since consumer hardware is primarily memory constrained.

Which is counter to the way most open weight models have been leaning. Many current models are MoE (Mixture of Experts) which by their very nature are parameter heavy. They are trading parameter efficiency for compute efficiency which is kind of the wrong tradeoff for current consumer hardware like gaming GPUs.

Re:Inkling 2 Mini by coder543 on Friday November 20, 2026 @05:18PM (#50583114)
[delayed]
Re:Inkling 2 Mini by Eisenstein on Friday November 20, 2026 @05:58PM (#50583556)
[delayed]
Re:Inkling 2 Mini by coder543 on Friday November 20, 2026 @06:04PM (#50583617)
[delayed]
Re:Inkling 2 Mini by cubefox on Friday November 20, 2026 @05:44PM (#50583404)
> Wonder what it means for safety and interpretability at this level of availability.

We already know from official model cards that the GPT-6 models are substantially less monitorable than other (non-looped) models. Which is not surprising, since they are doing some amount of Neuralese thinking between each token.

Re:Inkling 2 Mini by Almondsetat on Friday November 20, 2026 @04:55PM (#50582822)
What are some models in this ballpark that are also good at conversation? They always show off their software capabilities, but can it talk naturally? make creative stories?
Re:Inkling 2 Mini by hunterloftis on Friday November 20, 2026 @05:38PM (#50583344)
The closest to this that's also good at prose is gemma4 26b a4b. That's what I'm using for a fully-local "Tunnel Goons" (light D&D-style game) harness where multiple player agents and a DM agent improvise a story together.

I haven't tried inkling so I have no idea how it speaks, writes, and tells stories. I'd love to know what other small models folks are having success with in role-play, gaming, creative writing environments.

Re:Inkling 2 Mini by Almondsetat on Friday November 20, 2026 @05:43PM (#50583397)
Yeah, I've settled for gemma 4 too, but I'm always in search of a better model who gets what I'm asking
Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @10:45PM (#50585707)
Yes — Gemma writes in the most natural, least-annoying way.

The 12B model is super too; I think it might be a better writer than the 26B.

Getting it to talk about photographs and drawings is modestly entertaining and perhaps useful, though I haven't really found a use that is worth checking in on too regularly. It did point out some things about my own photos that were possibly valuable, though.

Re:Inkling 2 Mini by causal on Friday November 20, 2026 @05:03PM (#50582923)
Big fan of open weights but those Terminal Bench numbers are pretty abysmal, and I find that tends to track "agent usefulness" more than most SWE benchmarks.
Re:Inkling 2 Mini by rosstex on Friday November 20, 2026 @09:41PM (#50585379)
yeah I wanna see how well it does on an existing codebase
Re:Inkling 2 Mini by armcat on Friday November 20, 2026 @05:15PM (#50583086)
I didn't even know it was a thing to train using nvfp4 4-bit floats - I always thought it was a post-training quantisation thing. Fascinating!
Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @05:57PM (#50583544)
From memory, Nemotron Bolt is trained something like this?
Re:Inkling 2 Mini by sourcecodeplz on Friday November 20, 2026 @05:30PM (#50583265)
Even more talkative than Qwen
Re:Inkling 2 Mini by frou_dh on Friday November 20, 2026 @07:23PM (#50584350)
Re:Inkling 2 Mini by jakswa on Friday November 20, 2026 @05:37PM (#50583339)
The demo video was 32 tok/s and did a 15X speedup when generating the code? Weird that they didn't publish token speeds for this laptop setup they recorded. I'm worried this tri-loop setup is going to be too slow. QFN is impressive but lately been feeling a little brain-fried by its prose.
Re:Inkling 2 Mini by AtHeartEngineer on Friday November 20, 2026 @05:44PM (#50583412)
QFN was definitely distilled from Claude, the claudese prose is brutal.
Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @05:53PM (#50583503)
Quite an interesting model, but the second of my test prompts in "extra high" thinking in Unsloth really underscore what they say about knowledge here:

> While looping improves the model’s reasoning, it doesn’t enable its weights to store more facts—knowledge is mostly a function of total parameter count. Inkling 2 Mini’s score on AA-Omniscience, a measure of factuality, is in line with other models of the same size. Studying the variable-loop model, we saw that performances on long-context benchmarks improve with more passes, such as GraphWalks BFS and MRCR v2; but knowledge-intensive benchmarks doesn’t improve with more passes, such as AA-Omniscience, SimpleQA Verified, SimpleVQA, and WorldVQA.

> Inkling 2 Mini overcomes the knowledge gap by efficiently using search and tools. It leads open-weights models of similar size on BrowseComp, DeepSearchQA, and MCP Atlas. It is also strong on knowledge work benchmarks that incorporate search and tool use.

My prompt asked it to write a singleton class, single file plugin for WordPress that hooks user login and stores last login time as metadata. It has some code style requirements. The prompt also asked it to make no tool calls so it didn't search and only used its knowledge.

The model burned through 19200 thinking tokens on this task (which was the hard limit I had set) and didn't finish. Its thinking suggests it was going to make a mistake or two.

Both Qwen 3.6 35B and Gemma 4 26B can cope with this task, from memory, in far fewer tokens (even taking into account Qwen's tendency to loop).

Rerunning the same prompt with tool use allowed, it searched with browser use, pretty effectively, and got to a solution, but it still made the mistake the thinking traces suggested it would earlier (wrong hook name). Other than that it wrote good code.

But 20K-ish tokens in, I could then not get it to search to make corrections — it would simply EOT rather than actually run the tools it was saying it would call. Whether this is the model failing or Unsloth's harness and the model disagreeing, I don't know. Quitting and reloading the model didn't help.

It worked through my usual MySQL-5.0-compatible max score per category prompt nicely, though it didn't volunteer the suggestion of adding an index for speed, but it doesn't outperform LFM 2.5-24B-A2B, which is a non-looping model of similar size. It does work through SQL in its thinking traces quite nicely, though.

(I don't have an LFM comparison for the WP thing, I should re-download that and re-run it)

Car wash problem got a correct answer with a surreal caveat (why do they always seem to need to offer a caveat, no matter how weird)

So far, I think I would run Muse Glimmer or Nemotron Bolt over this; Muse Glimmer would output tokens more slowly but get to a result.

Re:Inkling 2 Mini by solarkraft on Friday November 20, 2026 @07:43PM (#50584492)
Interesting comparison to Muse Glimmer since that’s what I run currently (notably very token efficient).

> my test prompts in "extra high"

> The model burned through 19200 thinking tokens on this task (which was the hard limit I had set) and didn't finish

I still haven’t caught on to why people keep doing this. Benchmarks too. It feels like a masochistic exercise entirely removed from reality.

Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @08:38PM (#50584948)
Not sure what you mean. It defaults to “extra high” in the template as far as I can tell, I assume the benchmarks do too, so I tested against that, and for the most part it seems to be verbose but not insanely so (one imagines that looping compounds the higher reasoning).

Muse Glimmer and Qwen still finish that particular task in their highest reasoning levels without blowing through such a limit.

I will redo my testing at some point at lower effort.

But I would note again that even in extra high thinking mode, it went down a significant rabbit hole to try to make some code work off the basis of a “core” wordpress hook it had actually invented, and nothing I could do would get it to use the correct hook. When I told it about the correct hook what it actually did was engage in a series of tool calls to verify its original understanding, which of course failed to come up with results because the hook it was verifying did not exist. But it continued stubbornly, until it reached a point where it would blankly fail to call tools it said it would call.

I might retest with the Q8 version since it is not that much larger.

FWIW I run Muse Glimmer on “Reasoning strength: low”. It’s perfectly fine; I am not convinced xhigh makes that much difference.

I ultimately simply prefer the way Muse Glimmer works for the turn-by-turn assistant type usage I want; it thinks concisely, writes fairly well too. Struggling a little with tools for it in Pi, though.

Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @08:45PM (#50584993)
Additional note: looks like Unsloth has just updated for this model, so I am going to retest.
Re:Inkling 2 Mini by gregsadetsky on Friday November 20, 2026 @06:05PM (#50583627)
bonkers - I'm getting 20 tokens/s running the 4-bit version on my 2021 M1 Max MacBook pro with 64Gb...!!

recipe: https://gist.github.com/gregsadetsky/85e8b8ea91430dc826611b9...

Re:Inkling 2 Mini by meatmanek on Friday November 20, 2026 @07:57PM (#50584615)
I'm getting about 18 on the same hardware using the unsloth build of llama.cpp; I guess I should try upstream now that support is merged.

Theoretically with 400GB/s memory bandwidth and effectively 5.4B active parameters (accounting for the looping), token generation should be closer to 74 tokens/second. 18tps is close to what you'd expect for a dense model this size (22GB for UD-Q8_K_XL). Hopefully over the next few days, someone clever figures out what's going on and optimizes this.

Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @09:33PM (#50585324)
> Theoretically with 400GB/s memory bandwidth

Well it would be theoretical, alas — the M1 Max can't actually saturate the bus with the GPU.

I have seen varying suggestions that it might in practice between 320GB/s and 390GB/s; for the CPU it is much much lower (only 243 GB/s).

I am seeing 16-20 tokens per second generation in Unsloth Desktop.

Re:Inkling 2 Mini by jedisct1 on Friday November 20, 2026 @06:07PM (#50583653)
Doesn't seem to run in MLX-Serve yet :(
Re:Inkling 2 Mini by purplecats on Friday November 20, 2026 @06:09PM (#50583670)
hate these dark practices. they say click here to try it on our site. then u click. auth wall. fine you give it all your info. sunk cost is maximized. then they hit you with the credit card wall.
Re:Inkling 2 Mini by mentalgear on Friday November 20, 2026 @06:10PM (#50583683)
Somewhat astonished that thinking machines would do this considering that looping transformers a.k.a. neuralese is safety wise way harder to keep track of what the LLM is actually doing. by the way, thinking machines CEO is one of the people that quit openAi over safety concerns.
Re:Inkling 2 Mini by nunodonato on Friday November 20, 2026 @06:11PM (#50583695)
Thank you! Added to the Tiny League (https://tinyleague.nunodonato.com)
Re:Inkling 2 Mini by zergrush on Friday November 20, 2026 @06:17PM (#50583753)
pretty cool I guess I just don't know what i'd use it for other than being a nice toy. if it can fit on smaller embedded devices that could be useful
Re:Inkling 2 Mini by nitroedge on Friday November 20, 2026 @06:51PM (#50584068)
What is the marketing pitch and the one-liner for what makes this model special?
Re:Inkling 2 Mini by hatthew on Friday November 20, 2026 @06:52PM (#50584078)
Seems like this is the successor to Qwen 3.6 35B that we've been waiting for? I'm not aware of any other recent models of this size that can run well on mid-tier GPUs without severe quantization.
Re:Inkling 2 Mini by prometheus1992 on Friday November 20, 2026 @08:11PM (#50584731)
This is pretty good but also from their hf model release;

>> Note: each extra pass through the looped layers adds about 1.0B effective active parameters. Since the model runs with three passes, there are 5.4B effective active parameters per token.

Re:Inkling 2 Mini by mrboatshoepants on Friday November 20, 2026 @08:37PM (#50584937)
What coding harness would someone use with this, such as in the article’s demo video? The demo was impressive
Re:Inkling 2 Mini by easterncalculus on Friday November 20, 2026 @11:27PM (#50585942)
It looks like OpenCode in the video.
Re:Inkling 2 Mini by hedgehog on Friday November 20, 2026 @08:38PM (#50584949)
Seems pretty promising in a few quick coding tests. 27 tokens/s decode on Ryzen 395 and llama.cpp Vulkan. Besides the obvious unmerged PRs (32694, 32657, 32429, 32704) you'll want below tiny patch for about 30% speed boost.

Also, if anyone from the team is reading, if you have access for researchers I'd be interested in doing some higher volume tests & harness tuning than I have hardware for personally.

    --- a/src/llama-graph.cpp
    +++ b/src/llama-graph.cpp
    @@ -2670,10 +2670,12 @@ ggml_tensor \* llm_graph_context::build_moe_ffn(
                         const float beta = hparams.situ_beta;
                         const float lb   = hparams.situ_linear_beta;

    -                    ggml_tensor \* act = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, cur, 1.0f/beta)), beta);
    +                    // gate/up are strided views of the fused gate_up output; ggml-vulkan's SCALE
    +                    // needs a contiguous src, so without the cont both scales fall back to the CPU
    +                    ggml_tensor \* act = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, ggml_cont(ctx0, cur), 1.0f/beta)), beta);
                         act = ggml_mul(ctx0, act, ggml_sigmoid(ctx0, cur));
                         if (lb > 0.0f) {
    -                        up = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, up, 1.0f/lb)), lb);
    +                        up = ggml_scale(ctx0, ggml_tanh(ctx0, ggml_scale(ctx0, ggml_cont(ctx0, up), 1.0f/lb)), lb);
                         }
                         cur = ggml_mul(ctx0, act, up);
                         cb(cur, "ffn_moe_situ", il);
Re:Inkling 2 Mini by porkloin on Friday November 20, 2026 @09:39PM (#50585363)
Tested on a 7900xtx, I got a similar (~35%) improvement. Thanks for sharing!
Re:Inkling 2 Mini by shahbazac on Friday November 20, 2026 @08:54PM (#50585056)
I don’t understand the business model of these labs which obviously spend a huge amount of money on compute then release their models. I definitely glad they do, but how does it make business sense?
Re:Inkling 2 Mini by darksaints on Friday November 20, 2026 @08:56PM (#50585075)
I’m sure there’s tons of speculative money to be made via acquisition.
Re:Inkling 2 Mini by xrisk on Friday November 20, 2026 @08:58PM (#50585092)
it’s structured as a PBC. getting return on investments is the investor’s headache, not necessarily the company’s. we don’t know what deals have been struck.
Re:Inkling 2 Mini by edot on Friday November 20, 2026 @09:12PM (#50585191)
Kinda the same (directionally if not in magnitude) as companies that open source their core product (thus giving away millions of dollars in dev salaries, cumulatively up until release and also continuing for maintenance) but do managed deployments and consulting to pay the bills.

So, it’s advertising? For customers but also for any big companies who want to acquihire a really competent team and have them do the thing but in-house.

Re:Inkling 2 Mini by brainless on Friday November 20, 2026 @09:17PM (#50585223)
I think we will see a lot of small labs with funding. The funding amounts may be small but it is no secret that labs from around the world want to build usable (tiny/small/large) language models. Even if investors are not sure how or when the returns will come, they want to see traction and usage. Enough people have enough money to spend. It may sound odd but I think this is the largest spend of private wealth in research that we have seen globally. I see small investor willingness all over the place and I am far away from tech capitals.
Re:Inkling 2 Mini by HDThoreaun on Friday November 20, 2026 @09:25PM (#50585273)
Same model as companies like red hat. They hope to sign enterprise support deals for either managed services or bespoke updates
Re:Inkling 2 Mini by 80fj0348i on Friday November 20, 2026 @11:33PM (#50585974)
Traditionally governments invest in basic research because that's how they discover leads that enable civilizational jumps in technology. If you neglect particle physics research in the early 20th century you face existential threats to your sovereignty in the 1950s.

I can see large investors thinking in these terms - make lots of relatively cheap research bets to try to find the leads to tech that might make similar technological leaps within 15 years.

Re:Inkling 2 Mini by NiloCK on Friday November 20, 2026 @09:00PM (#50585113)
> a looped transformer, running the middle layers of the model three times for every token.

Completely unqualified take here, but unconditional re-runs of a fixed set of layers doesn't strike me as meaningfully 'loopy'. The exact performance profile is available in a more trad architecture by just inserting the layers after one another?

My presumption on loopy architectures was that they looped longer on token selections that were more difficult / higher stakes, with automatic kick-outs after some n loops to avoid machine analysis paralysis.

ALTHOUGH I guess that the presented architecture may also be useful insofar as it can fit in less memory than the same model with a tripled set of , and possibly that training over a looping architecture converges faster / truer than over naive architectures of the same size.

Re:Inkling 2 Mini by nsingh2 on Friday November 20, 2026 @09:08PM (#50585158)
I've been thinking about how small looped models are probably going to be the next big thing in local LLMs, and here it is. Exchanging a lower memory capacity requirement for more compute instead.

If the GPT-6.x models are looped, no doubt memory was a big reason.

Re:Inkling 2 Mini by dgacmu on Friday November 20, 2026 @09:40PM (#50585372)
M1 Pro Macbook, 32GB: About 10t/s using q4 xl while running browsers/etc. and interacting using the local browser.

test 1: write a python program to sort a csv by lastname and print out fields. Result: Nailed it. Nice python program. Nice explanation of output, if a little chatty (it gave an example output, which I didn't ask for).

test 2: my standard "identify the type and year of this coin" test. Failed miserably, I gave up after 12,000 tokens of it babbling incessantly about possible misinterpretations of a silver half-dollar. Unsurprising - most small models suck at this test - but slightly annoying it seemed to get in a loop.

Pretty good performance and result for the (very simple) code generation problem, though. One of the first models that seems OK running side-by-side with a browser on my older MBP.

Re:Inkling 2 Mini by dofm on Friday November 20, 2026 @09:48PM (#50585429)
What reasoning level in test 2?

Seemed to default to extra-high on Unsloth for me with Unsloth's Q4. (And yeah, it absolutely overran on a test I did.)

Re:Inkling 2 Mini by dgacmu on Friday November 20, 2026 @10:23PM (#50585599)
I just had "model default" selected; this was with unsloth Q4 also so I assume it's the same. That explains a fair bit about why it was so damn chatty generating the python. :)

Running it on low was better. It wanted to use tools which probably would have been good but couldn't get the in-browser js stuff to work on the image (perhaps it doesn't). It output a completely wrong answer (1979 dime instead of 1967 half-dollar) but at least it didn't loop.

Re:Inkling 2 Mini by chr15m on Friday November 20, 2026 @09:46PM (#50585418)
This looks so good. Might replace Muse Glimmer for me for local tool calling workloads. As good as Qwen 27b for many tasks and no state mandated censorship - yes please! Super excited to try this out today.
Re:Inkling 2 Mini by minimaltom on Friday November 20, 2026 @10:05PM (#50585500)
The thing I like about looped transformers is it implies theres some kind of modularity learned with how each iteration works on the residual stream, which sounds like a lovely inductive bias.

(all vibes, no data from me)

Re:Inkling 2 Mini by sodowork on Friday November 20, 2026 @10:10PM (#50585523)
So the transformer launched in 2017 is still the foundation of these current models
Re:Inkling 2 Mini by mannyv on Friday November 20, 2026 @10:14PM (#50585544)
I've been experimenting with local agent committees, and this should allow me to get a couple of agents running in "realtime" instead of sequentially.

It's funny, you think you have enough RAM then you realize that what you need is more memory bandwidth (ugh).

Re:Inkling 2 Mini by bigyabai on Friday November 20, 2026 @10:18PM (#50585565)
I tried warning Apple Silicon LLM adopters. You can buy a $500 GPU with memory bandwidth that is comparable to a $50,000 Mac Studio (and likely faster at GPU compute too).

If looped transformers on smaller LLMs is the future, then the VRAMlets win the war and the Unified Memory crowd overpaid for DDR5.

Re:Inkling 2 Mini by laybak on Friday November 20, 2026 @10:40PM (#50585673)
excited to see Thinking Machines accelerating their effort, but in their own way. loving this almost more indie flavor of AI research
Re:Inkling 2 Mini by mvkel on Friday November 20, 2026 @10:46PM (#50585711)
Models like this are a product of resource constraints. It's great. It's why I am for a data center moratorium. Scarcity breeds innovation. If we're forced to work with what we have, incredible efficiencies are on the horizon.
Re:Inkling 2 Mini by capv on Friday November 20, 2026 @10:57PM (#50585789)
In my tests inkling-2-mini shows results similar to GPT 5.6 with light reasoning. Impressive ...