Re:Physicists want to rebuild quantum theory from scratch
by f00_ on Monday September 04, 2017 @04:23AM
(#15166605)
Could you explain KL divergence and/or cross entropy to me?
from wikipedia: "In mathematical statistics, the Kullback–Leibler divergence is a measure of how one probability distribution diverges from a second expected probability distribution"
Kullback-Leibler divergence is used a lot in the reinforcement learning setting
"In information theory, the cross entropy between two probability distributions p and q over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set"
from wikipedia: "In mathematical statistics, the Kullback–Leibler divergence is a measure of how one probability distribution diverges from a second expected probability distribution"
Kullback-Leibler divergence is used a lot in the reinforcement learning setting
"In information theory, the cross entropy between two probability distributions p and q over the same underlying set of events measures the average number of bits needed to identify an event drawn from the set"
Cross Entropy is used as a cost function in neural networks rather than least squares http://neuralnetworksanddeeplearning.com/chap3.html#the_cros...