Just going back and look at GPT-2's output, it's amazing how much better this system is. It still doesn't "understand" anything, but the coherence of what it spits out has gone up drastically.

https://thegradient.pub/gpt2-and-the-nature-of-intelligence/