jelmervdl

Re:Firefox Translations doesn't use the cloud on Wednesday November 30, 2022 @09:08AM (#33800935)

You're pretty much right on the money. For ParaCrawl[1] (which I worked on) we used fast machine translation systems that were "good enough" to translate one side of each pair to the language of the other, see whether they'd match sufficiently, and then deal with all the false positives through various filtering methods. Other datasets I know of use multilingual sentence embeddings, like LASER[2], to compute the distance between two sentences.

Both of these methods have a bootstrapping problem, but at this point in the MT for many languages we have enough data to get started. Previous iterations of ParaCrawl used things like document structure and overlap of named entities among sentences to identify matching pairs. But this is much less robust. I don't know how they solve this problem today for low-resource languages.

[1] https://paracrawl.eu

[2] https://github.com/yannvgn/laserembeddings

Re:Firefox Translations doesn't use the cloud on Wednesday November 30, 2022 @08:07AM (#33800115)

We can already. Almost. The translator works perfectly* as just a web page[1]. It is only a matter of combining that bit with the full-page translation code[2] and some UI to toggle it.

It is a bit of a question whether this is the way to go. You're downloading about 20mb to get all the plumbing + translation models necessary to translate a page. It would be okay if it were widespread enough that we can assume everyone already has these in their browser cache, but the trend is moving away from that model of caching[3].

* given your browser support wasm SIMD. So an x86 processor with SSE4.1, although M1 also seems to work. And no Safari, because they haven't implemented wasm SIMD[4].

[1] https://translatelocally.com/web/

[2] https://github.com/jelmervdl/translatelocally-web-ext/blob/m...

[3] https://developer.chrome.com/en/blog/http-cache-partitioning...

[4] https://webassembly.org/roadmap/

Re:Firefox Translations doesn't use the cloud on Wednesday November 30, 2022 @06:14AM (#33799230)

The datasets these models are trained on are sentence pairs. So even if just a couple of sentences between two wikipedia sites are direct translations of each other, they will have appeared in the training set. They don’t have to have appeared on the same topic page, it could be that English Wikipedia has a whole category for a topic while Estonian Wikipedia has just a long single page, direct translations will still be identified and used in training.

I also think that the domain and the type of language used on Wikipedia is pretty consistent which will help a lot with unseen sentences.

By no means are these models bad! It’s just that Wikipedia is a particularly easy test for them.

Re:Firefox Translations doesn't use the cloud on Wednesday November 30, 2022 @05:58AM (#33799140)

That must be a bug. There is a lot of code in the extension specifically to make it track DOM changes and push translations for those as they happen.

React pages are especially tricky to translate as the framework does not take kindly on external JavaScript swapping out text nodes. There’s code in the extension to work around that.

Re:Firefox Translations doesn't use the cloud on Wednesday November 30, 2022 @03:56AM (#33798431)

It is running a full transformer type neural network on your cpu, it needs all the speed it can get.

It’s because the translation engine requires at least SSSE3.1 instructions[1]. These are translated to wasm SIMD instructions[2] which are only enabled by browsers if the underlying hardware is there to execute these at least somewhat efficient.

[1] https://github.com/browsermt/bergamot-translator/issues/418

[2] https://emscripten.org/docs/porting/simd.html

Re:Firefox Translations doesn't use the cloud on Tuesday November 29, 2022 @07:31PM (#33795342)

I don’t know about Disney, but in (European centred) machine translation we often pick Icelandic and Maltese as first languages to do specifically because they’re small but familiar enough as tests for the pipeline.

Re:Firefox Translations doesn't use the cloud on Tuesday November 29, 2022 @07:26PM (#33795306)

I think doing machine translation in the cloud makes more sense commercially (easier to monetise) and is a lot less challenging (better hardware, model sizes less of an issue, fewer worries about exposing proprietary code or models).

But I agree that this local experience should be the future. It’s good to have the control and independence.

Re:Firefox Translations doesn't use the cloud on Tuesday November 29, 2022 @07:12PM (#33795200)

https://translatelocally.com/

The website isn’t pretty but it’s the same tech as the browser extension. It came out of the same research project.

But because its optimised native code, and not limited by single threaded wasm, waaay faster!

Edit disclosure: I worked on that project, and this software.

Re:Firefox Translations doesn't use the cloud on Tuesday November 29, 2022 @07:04PM (#33795141)

It might be because Firefox Translate hooks into some private extension apis (for browser chrome and telemetry) that you can only use when your extension is signed by Mozilla itself. Maybe those apis have been removed from Librewolf, or it doesn’t accept Mozilla’s certificate as an exception.

I maintain a forked version of the Firefox extension[^1] that doesn’t use any private apis (or telemetry). It is slightly different[^2], but uses the same libraries.

Disclosure: I work for one of the project partners and contributed to the Mozilla extension.

[^1]: https://addons.mozilla.org/en-GB/firefox/addon/translateloca...

[^2]: https://github.com/jelmervdl/translatelocally-web-ext#differ...

Re:Firefox Translations doesn't use the cloud on Tuesday November 29, 2022 @06:55PM (#33795072)

To be fair, Wikipedia basically appears in its entirety in the training data. It’s a good test to see whether the translation model and all the plumbing works well, but not whether the model generalises well.

Re:Mozilla releases local machine translation tools as part of Project... on Thursday June 02, 2022 @06:25PM (#31601319)

I'm not aware of any actively maintained projects that give you this out of the box, but these two could be starting points for such a project.

Mozilla implemented a REST service based on (an earlier version of) bergamot-translator [1]. You could use that as a replacement for the WASM component in the addon's code.

I also know of some full-page translation demo code that uses the python bindings of bergamot-translator [2]. That's basically a web proxy a la Goole Translate.

Lastly, marian, the translation software that's being used, has a web server as well [3]. It does not support HTML though.

EDIT: see also my earlier comment for using it with Node or Python [4], which you could use to implement a simple web API.

[1] https://github.com/mozilla/translation-service

[2] https://github.com/jerinphilip/tagtransfer

[3] https://marian-nmt.github.io/docs/#web-server

[4] https://news.ycombinator.com/item?id=31599231

Re:Mozilla releases local machine translation tools as part of Project... on Thursday June 02, 2022 @04:41PM (#31600242)

All of them are freely available. Most of them through mtdata [1]. The exact list of the datasets is in the firefox-translations-training pipeline configuration file [2].

[1] https://pypi.org/project/mtdata/

[2] https://github.com/mozilla/firefox-translations-training/blo...

Re:Mozilla releases local machine translation tools as part of Project... on Thursday June 02, 2022 @03:31PM (#31599490)

I think the Firefox extension might not work on mobile because it hooks into some undocumented addon apis to draw that translation bar UI. Those might not be available on mobile.

The translation code itself should work on mobile. It's just some javascript & wasm (albeit with SIMD instructions not implemented in Safari's WASM vm…)

Re:Mozilla releases local machine translation tools as part of Project... on Thursday June 02, 2022 @03:07PM (#31599231)

The training pipeline is also on Github! [1]

I was experimenting with running the wasm version of bergamot-translator (the translation engine used by the addon) in node [2].

However, if you want more performance, using the Python library [3] or the native C++ interface [4] gets you further because the wasm build is limited to a single thread and thus a blocking interface, and can't use all the processor specific optimisations that are in the native builds.

EDIT: Another option is using translateLocally [5], which is a Qt desktop app based on bergamot-translator. It has a native messaging API that is designed as a much faster alternative to the wasm build for browser extensions, but it can also be used from Python [6].

[1] https://github.com/mozilla/firefox-translations-training

[2] https://gist.github.com/jelmervdl/a4c8b6b92ad88a885e1cbd51c6...

[3] https://colab.research.google.com/drive/1AHpgewVJBFaupwAbZq0...

[4] https://github.com/browsermt/bergamot-translator/blob/main/a...

[5] https://github.com/XapaJIaMnu/translateLocally

[6] https://github.com/XapaJIaMnu/translateLocally/blob/master/s...

Re:Mozilla releases local machine translation tools as part of Project... on Thursday June 02, 2022 @02:32PM (#31598797)

Extension Github page: https://github.com/mozilla/firefox-translations

Re:Mozilla releases local machine translation tools as part of Project... on Thursday June 02, 2022 @02:07PM (#31598446)

The upside of using WASM is that the extension itself can be easily ported to other browsers and platforms. The UI uses Firefox specific APIs but the parts that take the HTML from a page and push it through the translation engine would also work in any Chrome-based browser.

(Edit: also free sandboxing of a blob of C++ code that needs to handle arbitrary input from the web!)