News for hackers. Stuff that matters.
Re:Fine-tuned small LLMs can beat large ones with programmat... by littlestymaar on Monday August 04, 2025 @05:34PM (#44791620)
> AFAIK, distillation typically refers to tuning on the logits of the larger model

I think this is called “logit distillation” which is a particular form of distillation but not the only one.

> so you wouldn't be able to do that with fine-tuning APIs (OpenAI + Google in our blog post)

Dististillation from competitors' API is so common it has been given a name: it's called “distealing”.

There are no comments on this comment yet.