News for hackers. Stuff that matters.
Re:Small Language Models Are Also Few-Shot Learners by solarmist on Thursday October 14, 2021 @02:55PM (#28868407)
It's still millions of parameters. What kind if hardware is needed to train a 10 million parameter model?
Re:Small Language Models Are Also Few-Shot Learners by greens on Thursday October 14, 2021 @03:13PM (#28868653)
A single consumer GPU for ~hours
Re:Small Language Models Are Also Few-Shot Learners by binarymax on Thursday October 14, 2021 @03:55PM (#28869157)
Yes, verbatim from the paper: "Moreover, training with PET can be performed in several hours on a single GPU without requiring expensive hyperparameter optimization."
Re:Small Language Models Are Also Few-Shot Learners by solarmist on Thursday October 14, 2021 @03:56PM (#28869174)
Nice. Still reading it.
Re:Small Language Models Are Also Few-Shot Learners by solarmist on Thursday October 14, 2021 @03:56PM (#28869162)
Oh! Nice! The a much more accessible than I was expecting.
Re:Small Language Models Are Also Few-Shot Learners by solarmist on Thursday October 14, 2021 @03:10PM (#28868616)
Like could I train it in a week with a few GPU's? Or would this still require a cluster to train in a reasonable amount of time?