News for hackers. Stuff that matters.
Re:Gemma 2: Improving Open Language Models at a Practical Si... by canyon289 on Thursday June 27, 2024 @11:41AM (#40811684)
Hi, I work on the Gemma team (same as Alek opinions are my own).

Essentially instead of tokens that are "already there" in text, the distillation allows us to simulate training data from a larger model

There are no comments on this comment yet.