Re:Introduction to Machine Learning for Developers
by pineapple_sauce on Friday November 11, 2016 @05:01AM
(#12929319)
In the slides for unsupervised learning, what is meant by "Maximum Entropy"? Doesn't this just imply that the distribution will be uniform; i.e. it's no better than making a blind guess?
[ Parent
| on HN ]
Re:Introduction to Machine Learning for Developers
by machineman44 on Friday November 11, 2016 @05:17AM
(#12929400)
I have only seen a maximum entropy model as part of the supervised realm where it is a discriminative model. In other words, given some labeled data, we can draw a decision boundary. Maximum entropy in this context is almost certainly associated with the information theory definition, where the entropy of a collection of data based on the distribution of classes is measured. High entropy if each class is equally probable. Lower Entropy otherwise.