I like to think of this as adversarial training data. Adversarial inputs in general trick a NN to producing a specific output -- Adversarial training data tricks the NN into learning specific weights.

Note that the distilled data is not even from the same "domain" of input data any more. They're basically adversarial inputs.