> “If the linked article succeeds in linking any of the concepts of thermodynamics — which is one of the core building blocks of modern physics and relevant to almost every field of practical engineering — to DL models then that’d be huge boon. There are a lot of well honed techniques and tools available in the thermodynamics and statistical physics toolboxes which could vastly improve the ability of DL to be tuned for useful tasks in a scientific manner. ”

I think this is just wrong. Firstly, statistics has already been linked with thermodynamics for a long, long time, and people already have been using ideas about thermodynamic laws to describe potential functions and algorithms like Hybrid Monte Carlo and simulated annealing, even borrowing convergence metrics for things like parallel tempering.

When that connection is practical, it is about explicit, case by case algorithms, and not at all about recasting the generic interpretation of models into some other thermodynamic framework, which is just a type of cute toy analysis.

I would say specifically the example of Boltzmann machines highlights this exact point between a result that has pragmatic implications and a result that is just playing around with definitions.