sls56

yes tempay precisely, because we use English as our reference, there is bias in favour of languages similar to English. Additionally the bigger a language's wikipedia, the higher quality the word vectors, which also tends to improve performance

Cool rspeer, we will check NumberBatch out! Yes, as you say, our goal here is to show you can get something for nothing from pre-trained embeddings (I can learn the 78 matrices on my macbook in about ten minutes...).

The alignment procedure is completely agnostic to what word vectors you use. It only requires that cosine similarity is a good distance metric. We used fastText because there were lots of open-sourced vectors available...

People have actually tried to do things like this; though I suspect a linear transformation wouldn't work well. Chris Olah talks about it in his awesome blog: https://colah.github.io/posts/2014-07-NLP-RNNs-Representatio...

Yes definitely! We didn't want to complicate the repository, but from a few in-house experiments we already know that it is possible to improve the rotation matrices by: 1. First aligning to a reference language (English) 2. Then defining a new reference as the mean vector of all the languages for each entry in the training dictionary 3. Re-align the languages to this new reference "language" 4. Iteratively repeat 2 and 3 to convergence

As you suggest potentially this mean language is itself really high quality word vectors; but we haven't looked at this yet...

We haven't done this in a rigorous way yet no, but it would be really interesting! In particular, I wonder if it's possible to reconstruct the linguistic tree of languages evolving over time; without any prior knowledge, solely from the word vector spaces...