exgrv

We can! At Kyutai, we released a real-time, on-device speech translation demo last week. For now, it is working only for French to English translation, on an iPhone 16 Pro: https://x.com/neilzegh/status/1887498102455869775

We released inference code and weights, you can check our github here: https://github.com/kyutai-labs/hibiki

Samples: https://huggingface.co/spaces/kyutai/hibiki-samples

Inference code: https://github.com/kyutai-labs/hibiki

Models: https://huggingface.co/collections/kyutai/hibiki-fr-en-67a48...

Except it does? After Equation 2: "v_w and v'_w are the input and output vector representations of w."

This is probably due to our preprocessing of Wikipedia that did not get rid of all the '}' from the markup.

We decided to keep the casing, as it is useful for some applications such as named entity recognition.

Regarding the punctuation, as pointed out in another comment, these tokens might also be useful for some applications (and they are easy to filter out if you don't need them).

These models were trained in an unsupervised way, and thus cannot be used with the "predict" mode of fastText.

The .bin models can be used to generate word vectors for out-of-vocabulary words:

  > echo 'list of words' | ./fasttext print-vectors model.bin

or

  > ./fasttext print-vectors model.bin < queries.txt

where queries.txt is a list of words you want a vector representation for.

Hi, because we trained these vectors on Wikipedia, we released models corresponding to the 90 largest Wikipedia first (in term of training data size). More models are on the way, including Irish.

Regarding the size of the word vectors files: the text files are sorted by frequency, so it is possible to easily load the top k words only.

We might also release smaller models in the future, for training on machines without large memory.

These models were trained on Wikipedia.

It should be "Western Frisian" instead of "Western" (https://en.wikipedia.org/wiki/West_Frisian_language). Thanks for the catch!

Models are trained independently for each language. So unfortunately, you cannot directly compare words from different languages using these vectors.

If you have a bilingual dictionary, you might try to learn a linear mapping from one language to the other (e.g. see https://arxiv.org/abs/1309.4168 for this approach).

The graph algorithm described in the blogpost is more related to label propagation (which is more than 10 years old), than to "retrofitting". And the Google paper linked in the blogpost is citing the relevant literature correctly.

If the dataset is public, could you post a link? Otherwise, could you please send me an email? (My address can be found on the github README). Thanks!

Thanks for pointing this out. We design this library on large datasets and some static variables may not be well tuned for smaller ones. For example the learning rate is only updated every 10k words. We are fixing that now, could you please send us on which dataset you were testing? We would like to see if we have solved this.

It is possible to chose between negative sampling (ns), softmax or hierarchical softmax (hs) by using the -loss option.

At train time, the code supports multiple labels by sampling one of the k label at random. At test time, it only predicts the most probable label for each example.

We will add more functionalities for multi label classification in the future (predict the top k labels, etc...).

Except it is not the equality operator, but a special case of an indicator function (c.f. comment by pash).

A lot of people play the lottery. Only one person did this experiment. When you test multiple hypothesis (as in your lottery example), you need to perform a correction[2,3].

[1] https://xkcd.com/882/

[2] https://en.wikipedia.org/wiki/Multiple_comparisons_problem

[3] https://en.wikipedia.org/wiki/Bonferroni_correction

Members of the editorial board of scientific journals are not paid. So nothing prevents having a completely free and open access journal, which is peer reviewed. As a matter of fact, such journals already exist in some fields (e.g. the Journal of Machine Learning Research[1], which is the best journal in machine learning).

[1] https://en.wikipedia.org/wiki/Journal_of_Machine_Learning_Re...

Microsoft Research is also an excellent organization for CS related research.

There is this paper by Joulin & Mikolov: Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets (http://arxiv.org/abs/1503.01007).

In this case, the memory of the RNN is an ensemble of differentiable stacks.

I believe that the prestige of a journal is mainly defined by the members of the editorial board. So if you can convince members of the editorial board of prestigious journals to resign and join your new open access journal, it can very rapidly become very prestigious.

That is what happened in machine learning in 2001, when forty editors of the journal 'Machine Learning', published by Springer, resigned to join the open access 'Journal of Machine Learning Research', which was created in 2000. Today, JMLR is much more prestigious than Machine Learning (impact factor of 3.42 v.s. 1.46 in 2012).

See http://en.wikipedia.org/wiki/Journal_of_Machine_Learning_Res...

I also like this quote from Antoine de Saint Exupéry: "Il semble que la perfection soit atteinte non quand il n'y a plus rien à ajouter, mais quand il n'y a plus rien à retrancher", which can be translated as: "Perfection is finally attained not when there is no longer anything to add, but when there is no longer anything to take away".