This is a general problem for generative models, regardless of whether they are explicit latent density (variational autoencoders) or implicit density (GANs) or fully-observed (PixelCNN). To date no consensus has been reached in the research community on what the "right" way to evaluate a generative model is: e.g. Parzen Windows estimators, log-probability, ELBO, etc.

Yann Lecun himself (as of NIPS2016) was pretty critical of probability-based metrics, as those have strong dependence to the choice of model (e.g. if the model is poor the log-probability is meaningless).

In GANs, the critic and the generator are trained w.r.t. each other, reaching some kind of equilibrium. A recent proposal that seems to be "ok" for evaluating GANs was proposed by https://arxiv.org/abs/1705.05263, which is to train a separate critic on the generator, for use in evaluation (the generator never sees gradient information from this critic). This evaluation critic approximates the Wasserstein distance. One could imagine actually training the independent critic on a validation set of images not seen by the training set.