"A Neural Network for Machine Translation, at Production Scale"[1] says:
Machine translation is by no means solved. GNMT can still make significant errors that a human translator would never make, like dropping words and mistranslating proper names or rare terms, and translating sentences in isolation rather than considering the context of the paragraph or page. There is still a lot of work we can do to serve our users better. However, GNMT represents a significant milestone. We would like to celebrate it with the many researchers and engineers—both within Google and the wider community—who have contributed to this direction of research in the past few years.
That seems fair to me.
"Found in translation: More accurate, fluent sentences in Google Translate"[2] says:
Today’s step towards Neural Machine Translation is a significant milestone for Google Translate, but there’s always more work to do and we’ll continue to learn over time. We’ll also continue to rely on Translate Community, where language loving multilingual speakers can help share their language by contributing and reviewing translations.
"Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System"[3] says:
In September, we announced that Google Translate is switching to a new system called Google Neural Machine Translation (GNMT), an end-to-end learning framework that learns from millions of examples, and provided significant improvements in translation quality.
[1] https://research.googleblog.com/2016/09/a-neural-network-for...
[2] https://blog.google/products/translate/found-translation-mor...
[3] https://research.googleblog.com/2016/11/zero-shot-translatio...
This isn't cherry picking - you can try it yourself.
Randomly selecting the 2nd paragraph from https://www.hrw.org/fr/news/2017/01/02/israel/palestine-des-...
Human Rights Watch a documenté de nombreuses déclarations faites depuis octobre 2015 par des personnalités politiques israéliennes de haut rang, dont les ministres de la Police et de la Défense, appelant la police et les forces armées à tirer sur des individus suspectés d'attentats pour les tuer, avant même de déterminer si le recours à la force létale est, ou non, absolument nécessaire pour protéger des vies.
Choose which is automatically translated:
Human Rights Watch has documented numerous statements since October 2015, by senior Israeli politicians, including the police minister and defense minister, calling on police and soldiers to shoot to kill suspected attackers, irrespective of whether lethal force is actually strictly necessary to protect life.
vs
Human Rights Watch has documented numerous statements made since October 2015 by high-ranking Israeli politicians, including police and defense ministers, calling on police and armed forces to shoot at suspected terrorists even before deciding whether or not the use of lethal force is absolutely necessary to protect lives.
One of these in manually translated by an expert, one is by Google Translate (clicking on the "English" link on the page linked above will show the expert translation).
You can find other document collections to test here: http://www.miis.edu/academics/library/find/guides/translatin...
To honest it works better than I thought for French > English, but I when I tried some random French websites (not news) the results were not as good as this example and nowhere near human translations.
No, the "perfect score" is an "averaged" translation by multiple humans, measured by BLEU score.
But at the same time they say "It's true as well in my opinion that Google's Machine Translation is oversold" so the "they don't know what they're talking about except for all the points that I agree with them on" tone seems a bit silly.