Turing-NLG: A 17B-parameter language model

Re:Turing-NLG: A 17B-parameter language model by Tenoke on Monday February 10, 2020 @04:26PM (#22293555)

Any plans on training other (non nlp) huge models using ZeRO?

Specifically for Transformers - any plans to train a big model with a bigger context window?

Not that this one isn't very impressive, of course.

[ Parent | on HN ]

Re:Turing-NLG: A 17B-parameter language model by saurkt on Monday February 10, 2020 @07:24PM (#22294870)

Thanks for your kind words. Yes, we would like to next train a language representation model. And our hunch is that probably something which is a mixture of language representation and language generation would be able to get the best of both worlds.

[ Parent | on HN ]