I still remember sometime back Google's GNMT translate the Chinese text of 'I don't want to go to work' into English 'I want to go to work'. That example alone should be sufficient to showcase how most advanced machine learning model can fail at the simplest task.

It didn't understand the source material, it is just very good at memorizing and faking.

Whenever I attempt to use a translate site to translate more than a paragraph (Facebook or Google), it comes out a garbled mess - doesn't mean some sentences are seemingly clear and meaningful. The big thing is it chokes on idioms, not understanding, not leaving them as is but guessing some clearly wrong meaning. I occasionally find single-sentence posts on Facebook apparently translated astonishingly well, in the sense of being literate English and seeming to reflect the original meaning. But my French or Spanish is quite rough so translate could have missed something big - as I know it does when you get into longer texts.

Oh, that just means I happen to write in clear, unidiomatic English ;-). Add just a smidgen of irregular usage, contemporary metaphors and such and things can go South pretty quickly.

That'll just be a training problem (often translation is driven from example texts that have lots of translations, and we have many of these for multi-national orgs like EU, which necessarily don't include a lot of colloquialisms), and the inconsistency isn't found because the models built don't extend all the way out to real world experiences (training) and recognizing text as a real world experience narrative. I think a much deeper network could do better but we don't know how to train them, we take way too long as it is.

I've seen the evolution of translators and there is a big difference in results. We can try more convoluted examples.

Also do you think that every human can parse that contemporary metaphors better?

Statistically, but it is a shallow translation, with no modelling of what is said. This works astonishly well for translation, but gives the false intuition that it is meaningful; whereas it is in fact orthogonal to advancing in systems that comprehend a text so as to be able to reason their way out of winograd problems.

It's been "statistical" since about mid-00's. What's new is that it's now neural.