News for hackers. Stuff that matters.

A comment on Distilling Step-by-Step Outperforming Larger Language Models with Less Training. Parent comment

Re:Distilling Step-by-Step Outperforming Larger Language Mod... by gbasin on Thursday May 04, 2023 @10:11AM (#35815620)
you don't need access to the weights though, unless I missed something? this is akin to the various groups using GPT-3.5/4 to generate data for fine-tuning smaller models
[ Parent | on HN ]
Re:Distilling Step-by-Step Outperforming Larger Language Mod... by verdverm on Thursday May 04, 2023 @02:12PM (#35819748)
Sure, but that will run you up a hefty bill and add significant latency during training.
[ Parent | on HN ]
Re:Distilling Step-by-Step Outperforming Larger Language Mod... by gbasin on Thursday May 04, 2023 @02:13PM (#35819765)
The latencies and costs are changing so quickly...
[ Parent | on HN ]
Re:Distilling Step-by-Step Outperforming Larger Language Mod... by verdverm on Thursday May 04, 2023 @05:10PM (#35822117)
maybe for the model, but if I'm in GCP, then I am reaching out to another cloud, those network latencies and costs are pretty stable
[ Parent | on HN ]