Re:Phi-3 Technical Report
by HarHarVeryFunny on Tuesday April 23, 2024 @10:10PM
(#40139678)
They only experimented with different cosine learning rate decay schedules, but found results consistent across these, as well as across two different types of experiment where they either varied number of training tokens for a given model size, or varied model size for a given number of training FLOPs.
[ Parent
| on HN ]
There are no comments on this comment yet.