Thanks, I'll read that post now. I'm actually going by Yudkowsky's post:
http://lesswrong.com/lw/ul/my_bayesian_enlightenment/
It might not have been the exact same one, it was one where he mentioned the riddle and how his friend got the wrong answer because he was a frequentist. That might be where most of my misconception arises from.
Being horribly biased in favor of the Bayesian interpretation ever since I learned it was a thing I'll give an example of places that frequentists can be wrong. People who disagree can give counterexamples. ;)
http://lesswrong.com/lw/1gc/frequentist_statistics_are_frequ...
On the other hand, some argue that certain forms of inference are invalid and that it doesn't matter if they give the correct answer or not in practice because they're invalid. Calculus was attacked on this basis early on because many mathematicians thought that taking the limit of something as it approached 0 wasn't a thing you should be able to do.