Re:Bayes's Theorem: What's the Big Deal?
by jimrandomh on Tuesday January 05, 2016 @12:17PM
(#10844460)
> That is to say, if you start with any prior, will enough additional pieces of evidence always let you converge on the truth?
Yes, with two caveats. First, you can't have assigned zero probability to the right answer. Which means that if you have a probability distribution over hypotheses, and the right answer wasn't one of the hypotheses in the distribution, you're doomed from the start. The Solomonoff prior is a mathematical construct that gets around this problem by including every hypothesis that can be expressed as a Turing machine, but it gets used more as a philosophical token than an actual computing tool because it's unfeasible to use directly. The second issue is that if you have a bad enough prior, "enough additional pieces of evidence" can be an arbitrarily large amount, and there may only be a limited amount of evidence available to collect. In particular, rerunning an experiment over and over can only provide a limited amount of evidence, because of the possibility that some systematic error affects every instance of the experiment.
Re:Bayes's Theorem: What's the Big Deal?
by sawwit on Wednesday January 06, 2016 @09:24AM
(#10850599)
Ah I dimly remember that it also has an Occams razor built-in, so the prior probability falls off with the lengths of the strings that describe the Turing machines?
Yes, with two caveats. First, you can't have assigned zero probability to the right answer. Which means that if you have a probability distribution over hypotheses, and the right answer wasn't one of the hypotheses in the distribution, you're doomed from the start. The Solomonoff prior is a mathematical construct that gets around this problem by including every hypothesis that can be expressed as a Turing machine, but it gets used more as a philosophical token than an actual computing tool because it's unfeasible to use directly. The second issue is that if you have a bad enough prior, "enough additional pieces of evidence" can be an arbitrarily large amount, and there may only be a limited amount of evidence available to collect. In particular, rerunning an experiment over and over can only provide a limited amount of evidence, because of the possibility that some systematic error affects every instance of the experiment.