Re:Bayesian Generative Adversarial Networks
by carbocation on Thursday November 09, 2017 @11:39PM
(#15667982)
Is it fair to presume that we'll next see a variational inference approach to this, with faster but slightly less optimal results?
Next, the paper said: how about sampling the whole posterior distribution instead of finding only one point of maximum likelihood as classical GANs do. This is the point where the famous Markov chain Monte Carlo (MCMC) algorithms become useful. They use something called stochastic gradient Hamiltonian Monte Carlo, basically, it is a random walk algorithm, at each step, you follow a noisy gradient, as a result, you converge to the posterior distribution instead of a local minima as gradient descent does.
The paper claims that sampling the whole posterior helps to resolve problems with classical GANs. IMHO, this isn't a surprise claim, this is exactly what is good about Bayesian statistics.