Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Gemma 2: Improving Open Language Models at a Practical Size [pdf]. Parent comment
Essentially instead of tokens that are "already there" in text, the distillation allows us to simulate training data from a larger model
There are no comments on this comment yet.
Essentially instead of tokens that are "already there" in text, the distillation allows us to simulate training data from a larger model