News for hackers. Stuff that matters.

A comment on The Limitations of Deep Learning. Parent comment

Re:The Limitations of Deep Learning by yters on Monday July 17, 2017 @04:12PM (#14791922)
The data processing inequality says processing data does not increase its information content.
[ Parent | on HN ]
Re:The Limitations of Deep Learning by yorwba on Tuesday July 18, 2017 @01:17AM (#14794372)
But processing does increase the "obviousness" of the information content.

E.g. projecting the data onto independent dimensions doesn't change the information it contains, but it highlights that those dimensions are indeed independent. Decomposing a multimodal distribution into a mixture of unimodal distribution gives more insight than just viewing it as a bunch of data mushed together. And so on.

I think there should be a branch of information theory that quantifies the obviousness of information and how it is changed by various data processing methods.

[ Parent | on HN ]