Did you read the GPT-2 paper? Frankly the english examples therein are much more impressive than this, and this certainly seems within the realm of possibility for GPT-2 based on some of the other emergent behavior of the model (e.g. inadvertent french translation skills)