News for hackers. Stuff that matters.

A comment on Mistral NeMo. Parent comment

Re:Mistral NeMo by dannyw on Thursday July 18, 2024 @11:46AM (#40996786)
Anecdata, but I did some continued pretraining on a toy LLM using machine-translated data; of the original dataset.

Performance improved across all benchmarks; in English (the original language).

[ Parent | on HN ]
Re:Mistral NeMo by benmanns on Thursday July 18, 2024 @12:01PM (#40996977)
Am I understanding correctly? You look an English dataset, trained an LLM, machine translated the English dataset to e.g. Spanish, continued training the model, and performance for queries in English improved? That’s really interesting.
[ Parent | on HN ]