Making a neural network say “I Don’t Know”: Bayesian NNs using Pyro and PyTorch

Re:Making a neural network say “I Don’t Know”: Bayesian NNs ... by mlevental on Wednesday November 28, 2018 @08:53AM (#18551411)

how effective would this be in deep nets with small data sets?

edit: does anyone know of any papers that talk about this technique formally?

[ Parent | on HN ]

Re:Making a neural network say “I Don’t Know”: Bayesian NNs ... by paraschopra on Wednesday November 28, 2018 @09:12AM (#18551541)

Bayesian techniques are effective with small data sets if you build your model carefully. For example, if you are able to encode the generating process of your dataset, even a _single_ example could be used for effective training. For example, see this http://web.mit.edu/jgross/Public/lake_etal_cogsci2011.pdf

By providing a good generative model, you are shifting the effort from collecting more data to carefully coding your expert knowledge. In my view, this is an exciting paradigm as it allows combination of human knowledge (as generating process and prjors) that can be overcome (in posterior distributions) if enough data is accumulated against it.

[ Parent | on HN ]

Re:Making a neural network say “I Don’t Know”: Bayesian NNs ... by yuy910616 on Wednesday November 28, 2018 @09:05AM (#18551496)

interesting to see an answer too...usually bays is better with small data sets right?

[ Parent | on HN ]