Re:A Mathematical Theory of Communication (1948) [pdf]
by Q_is_4_Quantum on Friday May 01, 2020 @02:10AM
(#23039512)
It seemed clear to me at the time when I read (a few of - but especially https://bayes.wustl.edu/etj/articles/theory.1.pdf) ET Jaynes' papers on classical thermodynamics that they were commensurable. I'm mildly surprised there is some question about that!
Re:A Mathematical Theory of Communication (1948) [pdf]
by hackinthebochs on Friday May 01, 2020 @03:05PM
(#23045428)
The alphabet is to distinguish states. But this is the case in thermodynamic entropy as well, that each state is distinguishable. What is important is quantifying the set of possible states.
Re:A Mathematical Theory of Communication (1948) [pdf]
by wrnr on Friday May 01, 2020 @01:02AM
(#23039280)
vs an arbitrary defined number of dimensions?
Their are more exotic mechanisms to define entropy: https://en.wikipedia.org/wiki/Parastatistics
Re:A Mathematical Theory of Communication (1948) [pdf]
by dr_dshiv on Friday May 01, 2020 @01:23AM
(#23039362)
In a thermodynamic sense, information input (say, of photons) is generally going to increase the entropy of a receiving system. Only certain kinds of information can reduce the entropy of the receiving system. Too much information (i.e., too many photons) can literally burn your eyes.
I don't see how to square those manifest physical effects with the immateriality of Shannon information entropy.
Re:A Mathematical Theory of Communication (1948) [pdf]
by wrnr on Friday May 01, 2020 @01:38AM
(#23039415)
Is this not the problem of Maxwell's Daemon, if an intelligent agent observes and interferes with a particle system to reduce the system's entropy then overall entropy is still increasing because the agents needs to store and calculate information in some physical medium, be it brain matter of silicon.
Re:A Mathematical Theory of Communication (1948) [pdf]
by dr_dshiv on Friday May 01, 2020 @03:35AM
(#23039883)
Not exactly, I don't think.
In the Shannon formulation, there isn't a notion of "too much information" -- it is all symbols.
But in reality, too many photons can burn our eyes or a denial of service attack can break a website. Too much information is a reality --because we are thermodynamic entities.
I don't see how to understand that with Shannon.
In trying to connect Shannon entropy to thermodynamic entropy, I always get stuck on the fact that you need to have a defined alphabet or symbol set.