News for hackers. Stuff that matters.
Re:What Is Entropy? by glial on Monday July 22, 2024 @06:03PM (#41040293)
Can you explain this in more detail?

Entropy is calculated as a function of a probability distribution over possible messages or symbols. The sender might have a distribution P over possible symbols, and the receiver might have another distribution Q over possible symbols. Then the "true" distribution over possible symbols might be another distribution yet, call it R. The mismatch between these is what leads to various inefficiencies in coding, decoding, etc [1]. But both P and Q are beliefs about R -- that is, they are properties of observers.

[1] https://en.wikipedia.org/wiki/Kullback–Leibler_divergence#Co...

There are no comments on this comment yet.