A physicist argues that information is at the root of everything

Re:A physicist argues that information is at the root of eve... by yafujifide on Sunday April 25, 2010 @04:50PM (#1293512)

This article seems to imply that the author equates Shannon entropy with information. Is that so? It has always struck me that Shannon entropy is a good measure of information, but is not a good definition of information itself. Like the difference between a vector and the magnitude of a vector, information and its measure are not the same. What is information itself? If this author actually has a definition of that, I would like to see it.

[ Parent | on HN ]

Re:A physicist argues that information is at the root of eve... by Anon84 on Sunday April 25, 2010 @05:05PM (#1293535)

"You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage."

John von Neumann to Claude Shannon

[ Parent | on HN ]

Re:A physicist argues that information is at the root of eve... by jey on Sunday April 25, 2010 @07:16PM (#1293706)

How about http://en.wikipedia.org/wiki/Kolmogorov_complexity ?

The big problem with Shannon entropy is that it's measured with respect to a probability space, so it's not an "absolute" measure of information. Kolmogorov complexity "solves" this problem by defining information content of a string as the length of the shortest program that generates the string. (You still have to pick a reference UTM, but this only affects the program lengths at worst by a constant.)

[Disclaimer: Hobbyist, not expert.]

[ Parent | on HN ]

Re:A physicist argues that information is at the root of eve... by troystribling on Sunday April 25, 2010 @05:51PM (#1293596)

Shannon does not define information but the entropy of information.

[ Parent | on HN ]