News for hackers. Stuff that matters.
Re:A Straightforward Explanation of the Good Regulator Theorem by stanislavzza on Wednesday June 18, 2025 @07:02AM (#44308751)
This is pedantic, but I don't like the formulation of entropy as sum of p log(1/p). I think of log(p) as information of a single event, for which log base 1/2 gives the answer in bits. This makes the negative sign unnecessary, and technically all these formulas should specify the base of log > 1. Everything is cleaner with log base 1/2 (instead of e.g. using the equivalent negative log base 2). This comes up in log likelihood all the time too. I guess it's a prejudice against fractional bases.

There are no comments on this comment yet.