You could build frequentist models for the odds of 0 to 1 inches of rain falling in Cleveland tomorrow, 1 to 2 inches, and 0 to 2 inches, and the odds of 0 to 1 plus 1 to 2 don't need to add up to 0 to 2 inches. You can justify all 3 models, but they are inconsistent. Bayesian models don't allow that. I read about this in Aaron Brown's "Red-Blooded Risk". Here's an online source asserting "frequentists can have two different unbiased estimators under the same likelihood functions." https://chenghanyu.wordpress.com/2014/03/26/the-strengths-an...
Here's one saying that Bayesian models are self-consistent: http://bactra.org/notebooks/bayesian-consistency.html
Here's Eliezer Yudkowsky giving examples of my other point, that frequentist models can encourage sloppy behavior from scientists: http://lesswrong.com/lw/1gc/frequentist_statistics_are_frequ...
This is a core piece of your confusion. Infinity isn't a number; you can't multiply it by another number and get a meaningful result. You are saying you could run an infinite number of tests on Monday and then run an infinite number of tests on Tuesday and so on, but the concept of infinity holds that you did as many trials on Monday as you did on Monday and Tuesday combined. (Inf * 2 = Inf).
50 % probability for heads means that if you toss the coin infinitely often the relative frequency will converge to 50 %. But not quite, in very rare cases it won't. So you have to toss a coin infinitely often an infinite number of times and then you will see that all but a tiny fraction of the experiments indeed show the relative frequency converge to 50 %.
But now you have to quantify that this tiny fraction is something of probability zero. And even worse, it is still possible that none of your repeated experiments showed convergence, you seem right back where you started.
I would love to know to what you are referring with the inconsistent beliefs.
I would not say that Bayesian view is non-intuitive, I would say it fails to account for important things. A priori probabilities have to be rooted somewhere. Where does you beliefe in a 50 % probability for heads come from? Because you have previously observed the relative frequencies of coin tosses? Because you made some theoretical observations about symmetries?
There must be, at least so it seems to me, something about the probabilities associated with coin tosses that is independent of any individual, otherwise it would become rather difficult to explain how different individuals would arrive at similar probabilities independent of each other. So banning probabilities into the realm of beliefs does not cut it in my opinion.