Making a neural network say “I Don’t Know”: Bayesian NNs using Pyro and PyTorch

Re:Making a neural network say “I Don’t Know”: Bayesian NNs ... by sharemywin on Wednesday November 28, 2018 @08:00AM (#18551090)

found this online(might be useful):

Bayesian Approach: Pros and Cons

 	Pros

Can calculate explicit probabilities for hypotheses.

Can be used for statistically-based learning.

Can in some cases outperform other learning methods.

Prior knowledge can be (incrementally) combined with newer knowledge to better approximate perfect knowledge.

Can make probabilistic predictions.

 	Cons

Require initial knowledge of many probabilities.

Can become computationally intractable.

http://www.ru.is/faculty/thorisson/courses/v2008/gervigreind...

[ Parent | on HN ]

Re:Making a neural network say “I Don’t Know”: Bayesian NNs ... by paraschopra on Wednesday November 28, 2018 @08:23AM (#18551215)

> Require initial knowledge of many probabilities.

This is not a really big issue. You can always use uninformative priors, which is simply the Bayesian way of saying "I don't know" what the probability distribution looks like upfront. Then the data can help you infer the right shape of probabilities.

> Can become computationally intractable.

This is a bummer, but modern methods like variational inference and better sampling techniques are making Bayesian analysis on large-scale data and complex models tractable.

I linked a video the article that I highly recommend watching. Variational Bayes and Beyond: Bayesian Inference for Big Data https://www.youtube.com/watch?v=DYRK0-_K2UU

[ Parent | on HN ]

Re:Making a neural network say “I Don’t Know”: Bayesian NNs ... by nightcracker on Wednesday November 28, 2018 @10:19AM (#18552195)

> Require initial knowledge of many probabilities.

Can you make a two-stage design where the initial probabilities are generated by another machine learning model?

[ Parent | on HN ]