No, I think you are correct. I think the result for the CelebA dataset is a toy. But many results in this area are toys, e.g. deep dream.
You could exploit debayering artefacts.
In fact I wonder if any imaging sensor vendors run R&D trying to come up with novel neural net based debayering approaches - this could be a cheap way of bumping image quality/perceived resolution.
Do any datasets even exist where the images are at sensor pixel level?
That way the model would 'know' about imaging effects (I can't think of any specifically mechanical effects that could be in play here right this second) etc?
Or am I way off base here....