l_e_o_n

Edited for clarity.

I'm talking about the whole sequence; you're talking about the number of heads (or) tails in the sequence.

The number of heads is a sufficient statistic, so we'll get the same likelihood ratios out, but the likelihood values themselves will be larger.

You could make a similar point about the original CrossValidated Normal(0, 1)^N example by summarizing the data with the mean and sum of squares.

This doesn't work if the data were Cauchy(0, 1)^N instead.

You flip a possibly-biased coin 20 times and get half heads, half tails.

Under the model where the bias is 0.5—a fair coin—the probability of that outcome is (0.5)^20 or about one in a million. In fact, the probability of any sequence you could observe is one in a million.

Under the model where the bias is 0.4 the probability is (0.4)^20 × (0.6)^20 or about one in two million.

That is, the sequence we observed supplies about twice as much evidence in favor of bias = 0.5 as compared with bias = 0.4—this is likelihood.

Likelihood ratios are all that matter.

Morals:

- The more complex the event you're predicting (the rarer the tyical observed result) the smaller the associated likelihoods will tend to be

- It's possible that every observed result has a tiny probability under every model you're considering

- Nonetheless it makes sense to use the ratios of these numbers to compare the models

- This has nothing to do with continuity or logarithms