GPT2 does have a goal in mind, namely an idea akin to "minimal surprise". The problem is that it doesn't have a model of the world its talking about, so it quickly says things that are locally sensible but globally inconsistent.