[1] Actually it must not only be impossible to only ever get heads or tails, it must be impossible to obtain any sequence of outcomes that has not exactly the same number of heads and tails. There is only one sequence of outcomes with only heads and only one with only tails but there is already an infinite number of sequences with all heads but one tails or vice versa. And there is an enormous number of possible sequences with 40 % heads and 60 % tails or any other ratio other than 50/50.
Now, you might protest and say, "But this is a fair coin that just happened to land on heads arbitrarily many times". But then you are relying on a prior definition of probability in order to justify your objection to the frequentist's claim, since presumably what you mean by "fair" is that the probability the coin lands on heads is 1/2.
When tossing a coin infinitely often the ratio of heads and tails will converge to 50/50 but only in that sense that you have to toss the coin an infinite number of times an infinite number of times. And you can't really stop here because you may still deviate arbitrarily far from the expected outcome. So you toss the coin an infinite number of times an infinite number of times an infinite number of times. And you have to keep nesting your experiments until you made an infinite tower of experiments.
And I have no idea if that would be sufficient to ensure that you can no longer deviate from the expected outcome, my intuition tends towards no, but then again this is certainly far outside of the realm of things for which I would intuition expect to work. Likely I am really just confused and missing something obvious.
I think you mean "may not" rather than "can not". But yes, that's why you need to resort to limits[0] to understand the problem.
No. By my definition (lay, may be wrong), the probability is the ratio of heads to tails to which we converge at the limit (infinity). The only way for it to converge to 0 is if the probability of heads was 0 to begin with, which would contradict the initial assertion that it was a fair coin.
By the way, from your previous comment it seems to me that you're struggling with getting an intuitive grasp on arguments that rely on infinity. To most people (me included for sure) it's initially difficult to accept that infinity is qualitatively different than a finite amount. Our brain seems to superficially accept the concept, but really it keeps trying to reason about it as a really large (but still finite) quantity.
Are you familiar with the argument of whether 0.999... equals 1? (Spoiler: it does) Some people have a really hard time coming to terms with it, and the fundamental problem for many comes down to the same difficulty of reasoning about infinity. Also, it's a fun way to troll people :)
This seems problematic to me. You can not perform an infinite number of coin tosses and know to what the relative frequency converges. So how do you conclude that a fair coin has probability of 0.5 for heads then? I can't really put my finger on it, but that argument is somehow circular. The probability is what the relative frequency converges to and I can not get anything other than 0.5 because that means to probability would have to have been not 0.5 to begin with. I can really only say that I disagree, I just can't pin it down exactly.
Are you familiar with the argument of whether 0.999... equals 1? (Spoiler: it does) Some people have a really hard time coming to terms with it, and the fundamental problem for many comes down to the same difficulty of reasoning about infinity.
I am aware of that and it seems totally obvious to me. And I also certainly know that infinity is not just a really huge number, but I am not sure I internalized that well enough to not make any mistakes because of the difference. Actually I am pretty sure I make mistakes because of that.
I think in this case you have "convergence in probability" (but I've not read carefully the discussion). There is a stronger form, "almost sure convergence", where there is exact convergence with probability one (i.e. in some cases there is no convergence, but those happen with probability zero).
Edit: thinking more about it, I agree that the passage from mathematical probability to real-world probability is not very satisfactory. But it's not really surprising, the very notion of "real-world" is troublesome.
Just to clarify, I am talking about the frequentist's claim that in the long run the relative frequency becomes the probability, i.e. P(x) = lim[n -> ∞] nx / n where n is the number of trails and nx is the number of trails that yielded x. So again, if an infinite sequence of heads is a possible outcome, then that would mean P(heads) = 1 while the frequentist asserts that it necessarily must become 0.5.
IANAM, but in my understanding that's exactly what it means, if you express "in the long run" as "the ratio tends to 0.5 as the number of tosses tends toward ∞"