One way to look at these models is to say that they take raw input, convert it into a feature space, manipulate it, then output back as raw text. A nice example of this is neural style transfer, where the learnt features can distinguish content from style, so that the content can be remixed with a different style in feature space. I could certainly imagine evaluating the quality of those features on a scale spanning from rote-copying all the way up to human understanding, depending on the quality of the model.