News for hackers. Stuff that matters.
Re:What Is Entropy? by im3w1l on Wednesday July 24, 2024 @02:45PM (#41060373)
I hadn't realized that "differential" entropy and shannon entropy are actually different and incompatible, huh.

So the case I mentioned, where you know all the positions and momentums has 0 shannon entropy and -Inf differential entropy. And a typical distribution will instead have Inf shannon entropy and finite differential entropy.

Wikipedia has some pretty interesting discussion about Differential Entropy vs Limiting density of Points, but I can't claim to understand it and whether it could bridge the gap here.

Re:What Is Entropy? by kgwgk on Thursday July 25, 2024 @02:37AM (#41065394)
> So the case I mentioned, where you know all the positions and momentums has 0 shannon entropy

No, Shannon entropy is not applicable in that case.

https://en.wikipedia.org/wiki/Entropy_(statistical_thermodyn...

Quantum mechanics solves the issue of the continuity of the state space. However, as you probably know, in quantum mechanics all the positions and momentums cannot simultaneously have definite values.