News for hackers. Stuff that matters.
Re:Turing-NLG: A 17B-parameter language model by Tenoke on Monday February 10, 2020 @04:13PM (#22293431)
I expect we'll see some very interesting, very big models following it. I didn't dig too far into the code but the library looks very easy to use and will open up a lot of doors for people who have a few or a few thousand GPUs.
Re:Turing-NLG: A 17B-parameter language model by ghawkescs on Monday February 10, 2020 @05:03PM (#22293881)
I must be missing it, where did you find a link to the code?
Re:Turing-NLG: A 17B-parameter language model by Tenoke on Monday February 10, 2020 @05:08PM (#22293915)
The code for the distributed training library, not the model - https://github.com/microsoft/DeepSpeed/