State-of-the-art text classification with universal language models

Re:State-of-the-art text classification with universal langu... by timClicks on Wednesday May 16, 2018 @12:17AM (#17079978)

Hi Jeremy (& Sebastian of course), this is an amazing contribution to the state of the art!

I haven't read things in depth, but am curious - how do these models cope with out of vocabulary terms?

[ Parent | on HN ]

Re:State-of-the-art text classification with universal langu... by madisonmay on Wednesday May 16, 2018 @11:27AM (#17083156)

Initially embeddings of OOV terms are initialized to mean embeddings, and then the language model fine-tuning step allows the model to adjust these values.

See [18] here: https://github.com/fastai/fastai/blob/master/courses/dl2/imd....

[ Parent | on HN ]