Re:Unsupervised Machine Translation Using Monolingual Corpor...
by OscarCunningham on Thursday November 02, 2017 @12:58PM
(#15612129)
I wonder if this relies on the sentences of the corpora referencing similar concepts. I'd be interested to see if it worked on a corpus of modern English and a corpus of classical Latin or Greek.
Re:Unsupervised Machine Translation Using Monolingual Corpor...
by visarga on Friday November 03, 2017 @01:44AM
(#15616926)
> the model starts with an unsupervised naive translation model obtained by making word by word translation of sentences using a parallel dictionary learned in an unsupervised way
So they DID have a parallel dictionary at first. How they learned it is another problem.
Re:Unsupervised Machine Translation Using Monolingual Corpor...
by nl on Friday November 03, 2017 @12:49AM
(#15616774)
Temporal drift of concepts is definitely a thing - there's a good paper from Stanford's NLP group showing it in word embedding for English.
The interesting thing is that this model might automatically correct for it (although of course it couldn't translate things for which no concept exists).
Re:Unsupervised Machine Translation Using Monolingual Corpor...
by matheist on Thursday November 02, 2017 @01:44PM
(#15612518)
I believe they also add a loss term corresponding to translating A to B using last iteration's model, then translating that B-sentence back to A and comparing to the original sentence.
1- Train a system that translates language A sentences into a representation space, and can translate back from that space.
2- Train a second system that does the same, but with language B, onto the same representation space.
3- Train an adversarial system that tries to look at the representation space and identify which language the sentence came from, retraining the language translation systems to try to fool that recognizer. Retrain the models to try to not be recognized by this third system.
The best way to 'hide' from the recognizer is to have a very similar distribution, to make it use similar points in the representation space for the same concepts.
Brilliant stuff.