Self Supervised Learning in NLP

Re:Self Supervised Learning in NLP by antpls on Monday May 25, 2020 @06:01AM (#23299616)

Hello, number 9) doesn't say what the task is?

Also, I always wondered, do those methods work universally on all languages? For example Chinese, Korean and Japanese, with different alphabets.

[ Parent | on HN ]

Re:Self Supervised Learning in NLP by amitness on Tuesday May 26, 2020 @01:46AM (#23307811)

I've elaborated number 9. Please let me know if it's clear now.

Some of these methods do. For example, the tasks that power word vectors apply for many languages. https://fasttext.cc/docs/en/crawl-vectors.html

Masked Language Modeling has been applied to learn cross-lingual language models. Look into: https://arxiv.org/abs/1901.07291

[ Parent | on HN ]