News for hackers. Stuff that matters.

A comment on NLP's ImageNet Moment: From Shallow to Deep Pre-Training. Parent comment

Re:NLP's ImageNet Moment: From Shallow to Deep Pre-Training by zawerf on Monday July 09, 2018 @12:28PM (#17491424)
Do you have links for these pretrained models? The only one I am aware of is OpenAI's where they fine tuned a Transformer architecture for 1 month on 8 gpus:

https://blog.openai.com/language-unsupervised/

https://github.com/openai/finetune-transformer-lm

[ Parent | on HN ]
Re:NLP's ImageNet Moment: From Shallow to Deep Pre-Training by sebastianruder on Monday July 09, 2018 @12:45PM (#17491540)
You can find ELMo here: https://github.com/allenai/allennlp/blob/master/tutorials/ho...

And ULMFiT here: http://nlp.fast.ai/category/classification.html

[ Parent | on HN ]
Re:NLP's ImageNet Moment: From Shallow to Deep Pre-Training by cs702 on Monday July 09, 2018 @02:01PM (#17492146)
For those who don't know, Sebastian Ruder is a coauthor of the ULMFiT paper: https://arxiv.org/abs/1801.06146
[ Parent | on HN ]