Re:Boltzmann Encoded Adversarial Machines
by tlarkworthy on Thursday April 26, 2018 @02:46PM
(#16934447)
RBMs have a native probabilistic output (the output is a distribution you can slice), but vanilla neural networks don't (the output is a vector). Is that right?
It's best to say that an RBM is an undirected NN which models a probability distribution of some variables. You can sample from the distribution (which is a stochastic process). There are other NN models which use feed-forward NNs to do similarly --such as GANs and VAEs and others. The generation process is also stochastic, but the difference is that you sample a noise distribution and then feed that through the NN. In all cases the generated samples are still vectors.
RBMs are natural candidates for models which handle these kind of issues quite well. 1. Although people have done work trying to get GANs to work well with multimodal data, it's pretty kludgy. 2. GANs do not provide a means of inference (contrast VAEs which can satisfy this demand). 3. We have built a solid extension of RBMs to temporal models which work quite well.
However, as explained in this paper, stock RBMs have significant training issues. This paper attempts to improve the situation.