I'm going home -> Je rentre a la maison
Home sweet home -> La douceur du foyer (the translation is weird but the word foyer was expected)
I feel at home -> Je me sens chez moi (this one is particularly good, it didn't translate the word home directly)
Can you share your exemples where it fails?
Isn't that basically the same as the Winograd problem?
People often assume a very benign, civilized environment for AI. In ordinary life, human beings (on the internet or off) take an adversarial approach to other people modeling the logical framework behind an utterance. They either subvert it for amusement (trolling or comedy) or profit (politics, propaganda, sales), and a tremendous amount of effort goes into it, much of which is very effective.
As people have observed, it can be very easy to convince a human that a machine is intelligent, as with ELIZA. But if you violate the presumptions of trust that a machine is designed with, it's going to be surprisingly vulnerable. To be superior to humans, a machine would have to be able to fend off an intelligent person trying to undermine it, not just work when it is spoon-fed.
This is about where I am stuck. I'll start believing that we truly are on the cusp of a revolution as soon as I see Google Translate reliably knowing when to translate "home" into French as "domicile", "foyer", something those lines, or as "accueil."
Right now it seems to very frequently choose "accueil", which is generally wrong, except when you're talking about websites and software user interfaces. That it's biased so strongly toward that error speaks volumes about how critical semantics are to sorting out natural language, and also about how bad current NLP systems are at dealing with semantics.