> They pretend their system actually solves machine translation, for real, like AlphaZero solved Go.

The post you linked to makes no such claim. The strongest claim to, well, anything, is that it produces "reasonable" translations between language pairs it has never seen.

I mean this bit:

This inspired us to ask the following question: Can we translate between a language pair which the system has never seen before? An example of this would be translations between Korean and Japanese where Korean⇄Japanese examples were not shown to the system. Impressively, the answer is yes — it can generate reasonable Korean⇄Japanese translations, even though it has never been taught to do so. We call this “zero-shot” translation, shown by the yellow dotted lines in the animation. To the best of our knowledge, this is the first time this type of transfer learning has worked in Machine Translation.

Because what they claim to do is really out there- it is something that not even humans can do. If they could actually do it then they would have solved machine translation for real.

And they haven't - like I say above, they seem to have misinterpreted a model of English-to-many (other languages) as an interlingua. And it's not even a very good model of English-to-many.

So the difference between hype and reality is astronomic and I feel very well justified to say what I say.

I disagree. By definition, if it's gibberish no amount of human-level understanding of language will be able to glean meaning from it. You cannot extract meaning from random noise. This is better than random noise; in fact it's way better than what Google Translate used to manage a decade ago.

It's not gibberish. It's just very bad quality. But it helps me extract the meaning, which is not that far from what I do with some of the emails I receive in my day job -- trust me!

I'll ask you the same question I'm asking above: if that kind of language is not gibberish, then why are we not all trying to communicate like that?

To take this a step further- if your child was speaking in such a manner you would be very, very concerned. But with GT it's OK? Why?

This is absurd. You keep insisting on perfection or nothing at all. There is such a thing as incremental progress.

Google Translate is supremely useful in the absence of a better alternative. Nobody is contesting that human translation is better than Google Translate, but a human translator isn't always available. Google Translate is better than nothing at all.

A child's first many, many attempts at language are absolute gibberish, then slowly individual words, then combinations of words, then gradually more complex sentences etc -- gradually over several years. When you child first says "mama", the common response is excitement, not berating the child and anyone excited that "mama" isn't Wordsworth, and asking if just saying "mama" is acceptable, why aren't we all just communicating using single words all the time?

A child being able to express itself at all using language is better than it not being able to, and cause for celebration, and so too it is with Google Translate.

>> You keep insisting on perfection or nothing at all. There is such a thing as incremental progress.

I think we're talking at cross-purposes here a bit. I don't really care whether GT is useful as a tool. I care about whether the translations it offers are good translations or not. I think you're basically saying that, if it's good enough for you, then it's a good translation. However, this is not a good measure of the quality of a translation! To begin with, what's good for you is a sujective measure. Further, we wouldn't accept "it's good enough for me" for any other translation than a machine translation- and it's accepted for GT only because we understand the limitations of machine translation systems.

So basically my pointing out the limitations of machine translation systems has nothing to do with whether it's useful to you. It's an attempt to explain why the results of machine translation are objectively bad.

I'll agree if you want that it's very difficult to get to such an objective measure of goodness or badness. But that's actually a reason why it's so hard to do machine translation properly: because it's next to impossible to evaluate machine translation objectively, which in turn makes for a very poorly defined task.

So you can feel justified to say that GT is OK for you, because you can understand it. Others can say that it sucks because they can't. And Google can say it's a breakthrough in AI because it scores high on their formal tests, evaluating using BLEU score (or something equally arbitrary).

But none of those tells us how good or bad GT translation really is. So we're left with what Hofstadter does (and what I did here): eyballing it. And, eyballing it, you can see that it's making elementary, egregious mistakes that are not justified by the hype surrounding GT as a product.

Btw, we can expect a child to get better at using language because most children do. So far, no machine translation system has got a lot better than GT- and I don't even agree that GT has gotten any better over the years, or that it's even any better than earlier systems. It just happens to be Google's system and they advertise it as the best ever- but they can't actually, you know, prove this.

> if your child was speaking in such a manner you would be very, very concerned. But with GT it's OK? Why?

Because if, past a certain age, my child keeps talking this way, it's a sign of development problems. Google Translate is an algorithm and I don't worry about its mental health. The standards for software/machines and human children are different and I find nothing surprising about this fact.

I don't think anyone is claiming Google Translate "is OK" in the sense of "we're done here, we've done all we could, let's stop thinking about automated translation and declare it mission accomplished".

My point about child speech is that the kind of utterances produced by GT would not be acceptable as correct speech if they came from a child, let alone an adult. We'd recognise them as imperfect. Coming from GT though they're considered "good".

Two other comments accuse me of letting perfect be the enemy of good. But for this to happen, GT translations must be "good" in the first place. Well, they're not. They are often terrible.

I don't understand why this criticism is even controversial. Terrible won't become good if we don't point out that it is, in fact, terrible.

Plus, I'm very concerned about promoting commercial products as examples of technological and scientific progress. Of course Google has an incentive to claim its translation service is great. Why does anyone else?

> My point about child speech is that the kind of utterances produced by GT would not be acceptable as correct speech if they came from a child, let alone an adult. We'd recognise them as imperfect. Coming from GT though they're considered "good".

Because the standards for automated translation, something that didn't exist a few decades ago, are different from the standards of human speech and translation? No-one is saying Google Translate is on par or anywhere near human translation.

> Terrible won't become good if we don't point out that it is, in fact, terrible.

Yes, but you see complacency where there is none. The engineers who achieved this are rightfully proud about it, but I doubt they think this is the end of the line for automated translation.

It's helpful to say "this is flawed here and there, you can do better!", but your attitude seems needlessly antagonistic. Frankly, it sounds to me as if you feel threatened by this development. You shouldn't. As you point out, this is miles behind actual human translation in subtlety. Human-level understanding of speech (with double meanings, play on words, puns, understatements, etc) may very well be impossible for software to achieve; in fact, I think it's forever out of reach. But that's not the goal of automated translation; it's merely meant to be a useful tool when there are no human translators available.

> Plus, I'm very concerned about promoting commercial products as examples of technological and scientific progress. Of course Google has an incentive to claim its translation service is great. Why does anyone else?

Because they are definitely examples of technological progress and we can feel excited about it, and hope it gets even better! I couldn't care less about the Google brand. I understand your concern and I'm not excited about Google-the-business.

>> No-one is saying Google Translate is on par or anywhere near human translation.

Well, they kind of do. Like I say elsewhere, the common measure of machine translation performance is a family of metrics like BLEU or ROUGE that basically compare machine translation to human translation. So when a research team claims strong performance in machine translation, they're really saying that their systems can do the kind of translation that humans can do, in a very literal sense (of similarity between tokens or n-grams). Besides, like Hofstadter says, Google has often claimed its NLP systems (for example, its NL Parser, Parser Mc Parseface) approach or outdo human performance.

Google is quite unrealistic in the way it promotes its technology and claiming it does better than humans is its bread and butter in this regard.

>> Frankly, it sounds to me as if you feel threatened by this development.

I don't have any reason to feel threatened :) I'm a machine learning PhD researcher with some hands-on experience of NLP (though not machine translation specifically- I've only theoretical knowledge of it). I have a background in foreign languages and translation, but not professionally.

If I can summarise my concerns, it's not that I'm worried that machine translation will become so good it will take the jobs of human translators. I'm worried that we will keep replacing human workers with bad AI and end up making our lives a lot worse in the process.