Engineering Uncertainty Estimation in Neural Networks

Re:Engineering Uncertainty Estimation in Neural Networks by tony_cannistra on Tuesday April 16, 2019 @12:26AM (#19671127)

It's cool how "bayesian neural networks define a distribution over neural networks" –– so you can "sample from the posterior." Reminds me of the Gaussian Process learning framework, which seems quite similar (distributions over functions). Has anyone thought of this overlap?

edit: posted before reading other comments; seems like @syntaxing's link is what I'm looking for: "A network with infinitely many weights with a distribution on each weight is a Gaussian process"

[ Parent | on HN ]

Re:Engineering Uncertainty Estimation in Neural Networks by clircle on Tuesday April 16, 2019 @08:31AM (#19673052)

That a Bayesian NN "defines a distribution over neural networks" means that a Bayesian NN is a NN with priors on the weights. So it's not that there is some special similarity to GPs. BNNs are like any other Bayesian model, but now a NN is the likelihood.

The situation is usually flipped with Bayesian GP models -- a GP is usually used as a prior on the linear predictor.

[ Parent | on HN ]