Not quite; the probability of n/2 successes in n trials is given as Binomial(n,p) not p^n. p^n is correct for a single sequence but there are many possible sequences that result half heads, half tails and so you have a factor of "N choose X" or the so called "Binomial Coefficient".

> (0.4)^20 × (0.6)^20

and I think you mean (0.4)^10 × (0.6)^10 or more generally p^x*(1-p)^n-x.

I'm talking about the whole sequence; you're talking about the number of heads (or) tails in the sequence.

The number of heads is a sufficient statistic, so we'll get the same likelihood ratios out, but the likelihood values themselves will be larger.

You could make a similar point about the original CrossValidated Normal(0, 1)^N example by summarizing the data with the mean and sum of squares.

This doesn't work if the data were Cauchy(0, 1)^N instead.

> half heads, half tails.

> Under the model where the bias is 0.5—a fair coin—the probability of that outcome is (0.5)^20 or about one in a million.

No.

Edit: someone downvoted, ha. It's closer to 1 in 6.

1 in a million is the probability of correctly predicting a unique sequence of 20 coin flips, in the exact order. (E.g. first 10 flips heads, 2nd 10 flips tails, in that order - 1 in a million)

Edited for clarity.

I'm surprised people are conflating the Binomial distribution with OP's statement. He is talking about one specific outcome of half heads/half tails (where order matters). There is exactly one way to get that outcome.

You only knew that because you know how to do the math yourself, friend.

If you read what he said (I quoted it), he was not talking about one specific outcome. He didn't...well, specify that. He said half and half.

I also know how to do math, as I think I proved in my own comment. So I don't accept your insult.

You have given a nice clean answer that does not make any errors (such as talking about the likelihood as a density in parameters, which of course it is not). Thanks for writing it down.

The only other thing worth adding to what you have written is that the likelihood is a product of N factors.

As such, it will essentially always diverge toward infinity (if the density factors are on average greater than 1) or collapse fast towards zero (if the factors are on average less than 1, as in your example and in OP).

So this very structure (arising from the IID observations) implies that no “stable” density will pop out. It’ll always blow up or down!

One way to stabilize things is to take (1/N) times the log of the likelihood. Then you will indeed converge to something familiar - the entropy, - E log p(x).