An ImageNet-like text classification task based on Reddit posts

Re:An ImageNet-like text classification task based on Reddit... by samcodes on Monday October 29, 2018 @11:44AM (#18328406)

The post mentions not getting great results with OpenAI Transformer. I haven't tried that, but using a similar framework, ULM-FiT, I narrowly beat the fasttext benchmark on a 250-class dataset we use internally. I will follow up with how it does on this data set.

[ Parent | on HN ]

Re:An ImageNet-like text classification task based on Reddit... by sh33mp on Monday October 29, 2018 @12:58PM (#18329030)

ULM-FiT and OpenAI's Transformer* are quite different. Both are pretrained language-models, but ULM-FiT is a standard stack of LSTMs with a particular recipe for fine-tuning, whereas the OpenAI's Transformer uses the much newer Transformer architecture, and no really fancy tricks in the actual fine-tuning. I suspect the difficulty is with the Transformer model itself - this is not the first time I've heard that it is difficult to train.

* = To be clear, this refers to OpenAI's pretrained Transformer model. The Transformer architecture was from work at Google.

[ Parent | on HN ]

Re:An ImageNet-like text classification task based on Reddit... by gargarplex on Monday October 29, 2018 @12:38PM (#18328858)

Do any freelance work? We have a small fastText project. Email in profile if you're interested.

[ Parent | on HN ]

Re:An ImageNet-like text classification task based on Reddit... by sweezyjeezy on Tuesday October 30, 2018 @03:13AM (#18334448)

I'd be very interested to know, thanks!

[ Parent | on HN ]