News for hackers. Stuff that matters.
Re:A brief introduction to the beauty of Information Theory by dentalperson on Thursday May 07, 2020 @01:52PM (#23105902)
Shameless plug: I'm trying to start an online reading group for "Elements of Information Theory" by Cover and Thomas.

If you're interested, please join (https://www.reddit.com/r/mathreadinggroup/) and the discord (https://discord.gg/D9FjZXs).

Re:A brief introduction to the beauty of Information Theory by mikorym on Thursday May 07, 2020 @07:18AM (#23101740)
This is an interesting topic and is closely related to what I am working on at the moment.

One interesting thing I can say is that while subsets of a set are the language of probability and of logic (the existential quantifier and the universal quantifier are the two adjoints of the inverse image function), we have the dual to be the language of entropy: partitions of a set.

You can unify the two via a self-dual set theory, where you essentially replace the notions of subset (i.e., subobjects) and partitions (i.e., quotient objects) with an abstract subobject notion. This was part of the basis for my MSc and then my supervisor and a colleague of mine [1] showed that you can have what they call a Noetherian form over the category of sets. It is essentially by taking the pullback of the subobject and quotient object functors, though I can't remember if it is the usual pullback when seen as a category of categories.

[1] This is not all published, but you could have a look at the papers of Zurab Janelidze. The key point was his work on functorially self-dual group theory.

Re:A brief introduction to the beauty of Information Theory by tuxxy on Thursday May 07, 2020 @08:58AM (#23102433)
It's funny seeing this at the top of HN today. I just wrote a utility in Rust this past weekend to calculate the Shannon and metric entropy of a file (https://github.com/tuxxy/entropy/).

I really love the Shannon entropy equation. I don't know why, but I just find it so fascinating and elegant. I love it so much that I even got it tattooed on my wrist recently haha! (https://twitter.com/__tux/status/1257737842308022275/photo/1)

Re:A brief introduction to the beauty of Information Theory by birktj on Thursday May 07, 2020 @01:29PM (#23105632)
I like the original paper by Shannon "A Mathematical Theory of Communication" (http://people.math.harvard.edu/~ctm/home/text/others/shannon...) a lot. It is quite readable and also probably the most important paper in the field of information theory.
Re:A brief introduction to the beauty of Information Theory by jzer0cool on Thursday May 07, 2020 @06:05AM (#23101372)
I was introduced first to "gray code" as an undergraduate for error correction. For those new to this topic it could be where you can start for an introduction.