Re:TherML – Thermodynamics of Machine Learning
by kgwgk on Sunday July 15, 2018 @06:18PM
(#17537397)
My bad, I understood "more recent work in ml" as being recent in ml terms, not as work in ml which is more recent than the initial use (and abuse [1]) of thermodynamic concepts in information theory.
[1] Shannon, according to Tribus and McIrvine: "My greatest concern was what to call it. I thought of calling it ‘information’, but the word was overly used, so I decided to call it ‘uncertainty’. When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, ‘You should call it entropy, for two reasons: In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage."
[1] Shannon, according to Tribus and McIrvine: "My greatest concern was what to call it. I thought of calling it ‘information’, but the word was overly used, so I decided to call it ‘uncertainty’. When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, ‘You should call it entropy, for two reasons: In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage."