News for hackers. Stuff that matters.

A comment on Understanding Shannon's Entropy metric for Information [pdf]. Parent comment

Re:Understanding Shannon's Entropy metric for Information [pdf] by SAI_Peregrinus on Friday October 08, 2021 @02:24PM (#28802415)
Shannon entropy works for anything you can assign an alphabet of symbols to. It doesn't need the thing itself to be symbolic, just the description of the thing.
[ Parent | on HN ]
Re:Understanding Shannon's Entropy metric for Information [pdf] by dr_dshiv on Friday October 08, 2021 @03:39PM (#28803281)
That seems like a strong claim. Any proof?
[ Parent | on HN ]
Re:Understanding Shannon's Entropy metric for Information [pdf] by SAI_Peregrinus on Friday October 08, 2021 @08:51PM (#28805980)
Sure. https://people.math.harvard.edu/~ctm/home/text/others/shanno...

Particularly part 4, The Continuous Channel. No discrete symbols there!

[ Parent | on HN ]
Re:Understanding Shannon's Entropy metric for Information [pdf] by dr_dshiv on Sunday October 10, 2021 @04:14AM (#28816709)
Maybe I’m not reading it properly, but in part 5, where he deals with transmission of continuous signals he doesn’t use his entropy formulation. He uses a distance metric, showing the difference between the sent signal and the received.
[ Parent | on HN ]