News for hackers. Stuff that matters.
Re:The Kimi K3 Moment by milkshakes on Saturday July 18, 2026 @04:39PM (#48962141)
i never assumed that, and i do keep up with the publications. i'm also not saying it's a dumb thing to do! what i am saying is that empirically, it appears that distillation of a more advanced model is a required first step for them to train a borderline competitive, cheaper model. in effect, their training is subsidized by the frontier labs.

if this were not the case, then we would be observing chinese models that far surpass frontier models in capabilities, rather than "almost as good, but much cheaper", and we would be having a very different conversation. what happens to these efforts when the subsidy is cut off?

Re:The Kimi K3 Moment by InsideOutSanta on Saturday July 18, 2026 @04:50PM (#48962247)
> empirically, it appears that distillation of a more advanced model is a required first step

I see no evidence for that.

> if this were not the case, then we would be observing chinese models that far surpass frontier models

It's pretty clear that the primary reason for the difference is budget and compute availability. Chinese labs have at least an order of magnitude less money than Anthropic and OpenAI.

> what happens to these efforts when the subsidy is cut off?

They will continue making progress as they do now, minus the benefits of distillation.

Re:The Kimi K3 Moment by milkshakes on Saturday July 18, 2026 @05:15PM (#48962496)
https://www.anthropic.com/news/detecting-and-preventing-dist...

Moonshot AI Scale: Over 3.4 million exchanges

The operation targeted:

Agentic reasoning and tool use Coding and data analysis Computer-use agent development Computer vision Moonshot (Kimi models) employed hundreds of fraudulent accounts spanning multiple access pathways. Varied account types made the campaign harder to detect as a coordinated operation. We attributed the campaign through request metadata, which matched the public profiles of senior Moonshot staff. In a later phase, Moonshot used a more targeted approach, attempting to extract and reconstruct Claude’s reasoning traces.

Re:The Kimi K3 Moment by InsideOutSanta on Saturday July 18, 2026 @06:19PM (#48963017)
I'm assuming you posted that as evidence for the claim that "empirically, it appears that distillation of a more advanced model is a required first step", but I don't think it is. It's just evidence that Moonshot distills Anthropic's models, which, yes, they do.
Re:The Kimi K3 Moment by milkshakes on Saturday July 18, 2026 @06:34PM (#48963130)
it is not a required first step for training a model, sure. but that's not what i claimed. what i claimed is that is how they are so significantly _reducing the cost_ of training one! how else do you think they are doing it?
Re:The Kimi K3 Moment by idiotsecant on Saturday July 18, 2026 @06:10PM (#48962961)
>request metadata, which matched the public profiles of senior Moonshot staff

Translation: we have the machinery in place to identify our users, and actively do so.