Re:NLP's ImageNet Moment: From Shallow to Deep Pre-Training
by zawerf on Monday July 09, 2018 @12:28PM
(#17491424)
Do you have links for these pretrained models? The only one I am aware of is OpenAI's where they fine tuned a Transformer architecture for 1 month on 8 gpus:
Re:NLP's ImageNet Moment: From Shallow to Deep Pre-Training
by sebastianruder on Monday July 09, 2018 @12:45PM
(#17491540)
You can find ELMo here: https://github.com/allenai/allennlp/blob/master/tutorials/ho...
And ULMFiT here: http://nlp.fast.ai/category/classification.html
Re:NLP's ImageNet Moment: From Shallow to Deep Pre-Training
by cs702 on Monday July 09, 2018 @02:01PM
(#17492146)
For those who don't know, Sebastian Ruder is a coauthor of the ULMFiT paper: https://arxiv.org/abs/1801.06146
Re:NLP's ImageNet Moment: From Shallow to Deep Pre-Training
by romaniv on Monday July 09, 2018 @01:50PM
(#17492060)
>It is now possible to grab a pretrained model and start producing state-of-the-art NLP results in a wide range of tasks with relatively little effort.
Are there any applications/websites where this can be seen in action? It's increasingly hard to judge how good state-of-the-art really is from research papers.
Re:NLP's ImageNet Moment: From Shallow to Deep Pre-Training
by canada_dry on Monday July 09, 2018 @01:05PM
(#17491697)
5 yrs ago (mostly for fun) I tried out the 'state-of-the-art' - at the time - NLTK sentiment analyzer to correlate stock market changes with a variety of news/info sources.
I put it on the shelf because the sentiment analysis just wasn't up to snuf (i.e. the bias differentiation was too weak).
Might be time to try again!
It is now possible to grab a pretrained model and start producing state-of-the-art NLP results in a wide range of tasks with relatively little effort.
This will likely enable much more tinkering with NLP, all around the world... which will lead to new SOTA results in a range of tasks.