There's no need to introduce any special new concepts. It's simply a hierarchical graphical model. You have one variable which stands for p (distributed according to some prior, like a Beta distribution perhaps), and you have another (perhaps multiple), Bernoulli distributed variable whose parameter comes from the first variable and represents the actual coin value that you get.

If you do multiple coin flips, you have more of these Bernoulli variables but they all share a common parameter p.

One such example model is the Beta-Bernoulli process: http://www.math.uah.edu/stat/bernoulli/BetaBernoulli.html

> You have one variable which stands for p (distributed according to some prior, like a Beta distribution perhaps)

This prior sounds like the A_p distribution.

The point is it's a distribution, not a number. You can't just say "I think the probability of seeing heads is p". But if you say "I think the probability of seeing heads is distributed like this, which has a mean of p", then that captures everything you need to know about the coin to update your expectations when you see the results.

(In case it was unclear, A_p isn't a specific shape of distribution, or class of shapes of distributions, like beta or gaussian. It's a distribution over hypothesis-space, where the hypotheses are probabilities, and can have any valid distribution shape.)

I was suggesting that this is less "mysterious" than it seems to sound, once you use a hierarchical model, where one random variable's outcome is used as the parameter for another variable.

Imagine one random variable p at the top center of a page, and then several variables under it, each connected by an arrow coming from p.

The random variables down there are the coin flips (each results in 0 or 1), and they are identically and independently distributed with a Bernoulli(p) distribution. And p itself is a random variable which has a distribution (expressing how likely each p is).

I'm just clarifying that all this fits into normal probability theory, there is no need to introduce any sort of "meta-probability theory".

A biased coin usually means that one side has a greater chance of falling up, but they have both heads and tails, so p(H) + p(T) is still 1

I know that's what it means. I don't think they exist in the real world. See e.g. http://www.stat.columbia.edu/~gelman/research/published/dice... - I don't remember if I've read that specific thing, but I've read things that said the same.

(With skill, it's possible to bias a coin toss, but that's different.)

I was afraid somebody was going to bring up this paper. As the paper shows, they in fact do exist. But only if you spin rather than flip in the normal way.