Distilling Step-by-Step Outperforming Larger Language Models with Less Training

Re:Distilling Step-by-Step Outperforming Larger Language Mod... by anentropic on Thursday May 04, 2023 @03:55PM (#35821208)

But it's not 550M examples, it's 0.55M examples

Edit: just saw I wrote 550M in my original comment, even though I was thinking of it as the correct figure (550k)

[ Parent | on HN ]

Re:Distilling Step-by-Step Outperforming Larger Language Mod... by anentropic on Friday May 05, 2023 @05:46AM (#35827083)

Edit: obviously... also have to add the cost of training the T5-large on top of the LLM fees

[ Parent | on HN ]