I think it falls out of the way you pick R. If you pick from a distribution with a non-zero probability everywhere, then for any specific (non-equal) X and Y, you've got non-zero probability integrating between them.
Regardless of how A and B are chosen, if your R is not between them, then you break even; if your R is between them, then you win. So your advantage is entirely based on your probability of picking between A and B. As long as that probability is nonzero, then you have an edge.
The way you guarantee having nonzero probability is by choosing with a distribution that is positive everywhere (e.g the normal distribution). No matter where the interval [A,B] is, you have a chance of choosing within it.
Depending on the opponent's distribution vs yours, that chance may be very small, but it's still strictly greater than 0.
So the idea here is that the author's strategy is guaranteed not to lose, and may possibly win depending on factors that are not specified.
Of course you that could sacrifice a goat and examine the entrails and guarantee that you won't do worse than 50% and might do better.
If that R, A, and B are elements of the real line with A < B, What is the probability that A < R < B? It looks to me like the probability would be (B-A)/Inf = 0.
Huh? Sacrificing a goat (i.e. choosing randomly) will not do better than break even.
> If that R, A, and B are elements of the real line...
This makes assumptions about A and B that aren't in the puzzle. The fact that they're unknown doesn't imply that they're chosen from an infinite uniform distribution. It just means we have no information about them.
As the puzzle and strategy are stated, the methods for choosing A/B/R are all left unspecified. If we don't know how they're chosen, it stands to reason that we can't rule out the possibility of R falling between A and B, hence there's a nonzero chance of it happening. If you want a rigorous way to show this even for any given opponent strategy, the player can choose R from an unbounded distribution - which is nonzero everywhere, and therefore nonzero between A and B, for any choices of A!=B.
E.g. pick a number uniformly at random between 0 and 1. It might be rational, but the probability of that is 0.
The opponent's strategy is unknown and unspecified, per the puzzle. The point of the article is that there is a strategy that never does worse than chance, and may do better, depending on the opponent's choice of numbers. Sure, the opponent can make your advantage arbitrarily small - the article says that directly. But the notion that they are trying to do so is not part of the puzzle, and finding a strategy that prevents them from doing so is not what the author claims to have done.
No, that's not even a thing.
Consider a special case of the experiment: One in which the number picker always picks numbers within a finite interval; i.e., for numbers E1 and E2 inside the envelopes |E1 - E2| < X for some X. In this case, case 3 occurs with zero probability. There is an infinite interval over which both E1 and E2 are greater than any random number R. Similarly, there is an infinite interval over which both E1 and E2 are less than any given random number R. But there is only a finite interval over which it is possible for E1 and E2 to be on opposite sides of R.
Here's another way to think of it: You have an infinitely large dartboard. The author wants to paint a finite sized bullseye on the dartboard. The size of the dartboard is the distance between E1 and E2 (the two numbers in the envelopes). He throws a random dart at the dartboard and if he hits the bullsye he wins. The size of that bullseye is the "advantage" over random chance. But it doesn't matter how large you paint the bullseye, if it's a finite size then there's a zero probability that a random (finite sized) dart will hit the bullseye. The ratio of the area outside the bullseye to the ratio inside the bullseye is infinity.