It has to do with calculus. If the probability of a given result approaches zero as the result itself approaches positive or negative infinity, then the area under the curve approaches 1.

Imagine 1/2 + 1/4 + 1/8 ... to infinity. The sum approaches 1 as the denominator approaches infinity. With calculus, we can determine that with mathematical methods.

I think he was referring specifically to an unbounded uniform distribution, which is indeed not well-defined.

It's more of a philosophical statement than an actual implementation. Mainly that frequentist analysis begins with the prior "I dunno, could be anything..."

See also: The calculus answer.

It turns out that priors don't actually need to be distributions all the time. This often occurs when seeking maximally "uninformative" priors.