News for hackers. Stuff that matters.
Re:MAI Code 2 and Code 2 Flash by origini on Tuesday November 17, 2026 @02:42PM (#50537064)
I would counter argue with "But chinese models use distillation of Frontiermodels..." which is different than being a Frontierlab or not clear how different it is.

But Microsoft coming from the side like this might be very different if it holds up.

Re:MAI Code 2 and Code 2 Flash by adrian_b on Tuesday November 17, 2026 @02:52PM (#50537174)
For now I have not seen any credible evidence for "But chinese models use distillation of Frontiermodels...".

Of course, I would not surprised if they also do this, mainly because after OpenAI and Anthropic have succeeded to gather most training information that was easily accessible in the past and now it has become much more difficult to do the same. For some information that was hoarded by OpenAI and Anthropic public access may have become impossible otherwise than through their LLMs.

Nonetheless, distillation cannot be the main source of improvement for the Chinese models. While OpenAI and Anthropic provide negligible information about their models, the Chinese AI companies are much more transparent about their methods and we can see that there is a lot of diversity among their models and a lot of innovations with each new version.

Re:MAI Code 2 and Code 2 Flash by origini on Tuesday November 17, 2026 @03:00PM (#50537274)
I don't think there is perfect proof, but its taken as truth.

Indication for this are for example that Chinese models think they are another model: "chinese model think its claude".

My assumption is, that Chinese model found things Frontierlabs already had but didn't share. There was this paper were i had the feeling that this was a real and significant Research Result from Deepseek "DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression" but otherwise its just unclear.

The main issue is, that as long as it seems plausable that they do distillation, its hard to determine how big the gap is.

Not saying they can't catch up but they are also potentially limited by resources (money & nvidia GPUs)

Re:MAI Code 2 and Code 2 Flash by rbehrends on Tuesday November 17, 2026 @03:19PM (#50537587)
> Indication for this are for example that Chinese models think they are another model: "chinese model think its claude".

There is evidence (in particular from the analysis of reasoning traces [1]), but what models claim about their identity is pretty much worthless as evidence. LLMs are not self-aware. These claims come from whatever is in their training data, and Claude/GPT output including this is all over the web, including in some huggingface data dumps. Consider that people got Opus claiming to be Qwen or DeepSeek when asked in Chinese, and you'll see how weak this particular evidence is.

[1] E.g. https://arxiv.org/abs/2608.09867

Re:MAI Code 2 and Code 2 Flash by cmrx64 on Tuesday November 17, 2026 @05:08PM (#50538932)
So you’ve been completely ignoring all the work on introspective awareness, situational awareness, training set ordering awareness, subliminal learning, etc? Like it’s ok to be ignorant about the state of knowledge but you should at least correct your flawed mental model.
Re:MAI Code 2 and Code 2 Flash by quixoticelixer- on Tuesday November 17, 2026 @06:24PM (#50539662)
The thing that makes me think they do it is that you can tell there is more of a difference in personality between Gemini and Opus for example, than between Opus and most other models
Re:MAI Code 2 and Code 2 Flash by nzd on Tuesday November 17, 2026 @03:04PM (#50537333)
Heck, they destroyed some rare physical books after scanning them. Something to show their mentality.
Re:MAI Code 2 and Code 2 Flash by rbehrends on Tuesday November 17, 2026 @03:11PM (#50537464)
Yeah, there is a limit to how much you can get from capturing reasoning traces. (Traditional distillation of weights is not possible via the API.)

Patrick Toulme noted a while ago that the biggest advantage of that in his opinion came from bootstrapping the post-training process: https://xxcancel.com/PatrickToulme/status/206921157543762774...

At Thinking Machines, they said that they used open-weights models for bootstrapping post-training, including Kimi K2.5: https://thinkingmachines.ai/news/introducing-inkling/

Speaking of Thinking Machines, that's another data point. Their Inkling Small 2 model is just a bit behind DeepSeek V4.1 Flash 1017 in benchmark, and in my tests, that seems to be accurate, but mostly because they used less training data for pre-training, so they have less of a knowledge base, while similar reasoning capabilities. (One of the most interesting parts of Inkling Small 2 is that their training must have been very cost-efficient, apparently using less, but higher quality training data.)

And we're now seeing more and more Western near-frontier models from labs other than Anthropic and OpenAI to question whether distilling reasoning traces is really much of a difference maker.

Personally, my theory is that the gap is primarily due to Chinese labs having a harder time getting the necessary compute (especially top of the line GPUs) for training. These are, as far as I can tell, orders of magnitude differences in availability, so it wouldn't be surprising that a large compute gap is reflected in capabilities (and also indirectly visible through model sizes).

Re:MAI Code 2 and Code 2 Flash by eli on Tuesday November 17, 2026 @03:12PM (#50537497)
Even if OpenAI/Anthropic remain "a few months" ahead of open weight models indefinitely that still is not great news for them.
Re:MAI Code 2 and Code 2 Flash by giancarlostoro on Tuesday November 17, 2026 @03:34PM (#50537791)
All it would take is Nvidia finally releasing the 6090's instead of trying to milk enterprise GPUs. I don't understand the blind love for Jensen, he's clearly been screwing over all the consumers who made him and Nvidia wealthy.
Re:MAI Code 2 and Code 2 Flash by LeBit on Tuesday November 17, 2026 @04:16PM (#50538312)
Every week I ask ChatGPT an update on when the 6090 is going to get released.

Don’t hold your breath.

Re:MAI Code 2 and Code 2 Flash by origini on Tuesday November 17, 2026 @03:39PM (#50537854)
I currently do not bother with chinese models. I try them out, i use the small ones for small experiments but the price point difference matters so little that I want 'the current best'.

And 6 month feels still very different.

Its for sure an option or might have some kind of impact but if you have some kind of restrictions on what you should use and the price is not that big but the quality and possibilites are better, I can see companies just staying with Anthropic or OpenAI.

You also don't just switch out a model, they might need prompt updates etc.

But we will see I don't mind competition

Re:MAI Code 2 and Code 2 Flash by pseudosavant on Tuesday November 17, 2026 @05:35PM (#50539259)
That isn't countering though. Pointing out that distillation makes it "easy" for other labs to catch up, in a matter of months, supports the lack of a moat.

It's like they're runners drafting behind the lead runner. It is easier to run in their wake. If the lead wants to stay in front, they have to keep running into the wind.

Re:MAI Code 2 and Code 2 Flash by outside1234 on Tuesday November 17, 2026 @05:44PM (#50539335)
Do they need to stay in front though? Because if it is 25% the cost, I'm happy to be one leg behind the leader.
Re:MAI Code 2 and Code 2 Flash by epolanski on Tuesday November 17, 2026 @06:35PM (#50539774)
The distillation argument is stupid in any case and only people that don't understand training, post training or RL can make such asinine arguments.