News for hackers. Stuff that matters.
Re:MAI Code 2 and Code 2 Flash by adrian_b on Tuesday November 17, 2026 @02:52PM (#50537174)
For now I have not seen any credible evidence for "But chinese models use distillation of Frontiermodels...".

Of course, I would not surprised if they also do this, mainly because after OpenAI and Anthropic have succeeded to gather most training information that was easily accessible in the past and now it has become much more difficult to do the same. For some information that was hoarded by OpenAI and Anthropic public access may have become impossible otherwise than through their LLMs.

Nonetheless, distillation cannot be the main source of improvement for the Chinese models. While OpenAI and Anthropic provide negligible information about their models, the Chinese AI companies are much more transparent about their methods and we can see that there is a lot of diversity among their models and a lot of innovations with each new version.

Re:MAI Code 2 and Code 2 Flash by origini on Tuesday November 17, 2026 @03:00PM (#50537274)
I don't think there is perfect proof, but its taken as truth.

Indication for this are for example that Chinese models think they are another model: "chinese model think its claude".

My assumption is, that Chinese model found things Frontierlabs already had but didn't share. There was this paper were i had the feeling that this was a real and significant Research Result from Deepseek "DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression" but otherwise its just unclear.

The main issue is, that as long as it seems plausable that they do distillation, its hard to determine how big the gap is.

Not saying they can't catch up but they are also potentially limited by resources (money & nvidia GPUs)

Re:MAI Code 2 and Code 2 Flash by rbehrends on Tuesday November 17, 2026 @03:19PM (#50537587)
> Indication for this are for example that Chinese models think they are another model: "chinese model think its claude".

There is evidence (in particular from the analysis of reasoning traces [1]), but what models claim about their identity is pretty much worthless as evidence. LLMs are not self-aware. These claims come from whatever is in their training data, and Claude/GPT output including this is all over the web, including in some huggingface data dumps. Consider that people got Opus claiming to be Qwen or DeepSeek when asked in Chinese, and you'll see how weak this particular evidence is.

[1] E.g. https://arxiv.org/abs/2608.09867

Re:MAI Code 2 and Code 2 Flash by cmrx64 on Tuesday November 17, 2026 @05:08PM (#50538932)
So you’ve been completely ignoring all the work on introspective awareness, situational awareness, training set ordering awareness, subliminal learning, etc? Like it’s ok to be ignorant about the state of knowledge but you should at least correct your flawed mental model.
Re:MAI Code 2 and Code 2 Flash by quixoticelixer- on Tuesday November 17, 2026 @06:24PM (#50539662)
The thing that makes me think they do it is that you can tell there is more of a difference in personality between Gemini and Opus for example, than between Opus and most other models
Re:MAI Code 2 and Code 2 Flash by nzd on Tuesday November 17, 2026 @03:04PM (#50537333)
Heck, they destroyed some rare physical books after scanning them. Something to show their mentality.
Re:MAI Code 2 and Code 2 Flash by rbehrends on Tuesday November 17, 2026 @03:11PM (#50537464)
Yeah, there is a limit to how much you can get from capturing reasoning traces. (Traditional distillation of weights is not possible via the API.)

Patrick Toulme noted a while ago that the biggest advantage of that in his opinion came from bootstrapping the post-training process: https://xxcancel.com/PatrickToulme/status/206921157543762774...

At Thinking Machines, they said that they used open-weights models for bootstrapping post-training, including Kimi K2.5: https://thinkingmachines.ai/news/introducing-inkling/

Speaking of Thinking Machines, that's another data point. Their Inkling Small 2 model is just a bit behind DeepSeek V4.1 Flash 1017 in benchmark, and in my tests, that seems to be accurate, but mostly because they used less training data for pre-training, so they have less of a knowledge base, while similar reasoning capabilities. (One of the most interesting parts of Inkling Small 2 is that their training must have been very cost-efficient, apparently using less, but higher quality training data.)

And we're now seeing more and more Western near-frontier models from labs other than Anthropic and OpenAI to question whether distilling reasoning traces is really much of a difference maker.

Personally, my theory is that the gap is primarily due to Chinese labs having a harder time getting the necessary compute (especially top of the line GPUs) for training. These are, as far as I can tell, orders of magnitude differences in availability, so it wouldn't be surprising that a large compute gap is reflected in capabilities (and also indirectly visible through model sizes).