This isn't an NLP problem, it's an AI problem. Despite all the hype, we haven't actually made any progress toward solving strong AI, human-like general intelligence. NLP, computer vision, and many other AI problems are probably AI-complete, meaning that solving them entails solving strong AI. And the inability to generalize is exactly what separates the partial solutions we currently have from strong AI.