Link Between Deep Neural Networks and the Nature of the Universe

Re:Link Between Deep Neural Networks and the Nature of the U... by smaddox on Saturday September 10, 2016 @04:15PM (#12470707)

Pruning works for understanding sparse networks, but not for holographic networks, which can encode a much higher information density. Understanding a holographic network requires a large amount of effort, similar to determining the content of an optical holograph. You can try to develop a model, but if the information density is very high, it's no faster than observing the response to various inputs.

The problem becomes even more difficult if there is no clear distinctions between layers in the network, as in the brain.

[ Parent | on HN ]

Re:Link Between Deep Neural Networks and the Nature of the U... by rdlecler1 on Sunday September 11, 2016 @03:10PM (#12475048)

A network should be no more complex than it needs to be. If you're not pruning spurious interactions then you're implicitly assuming that each interaction is information bearing. If you're not pruning spurious interactions from holographic networks then you're assuming that they're all functionally relevant. I promise you they're not.

You're also assuming that holographic networks are somehow necessary for most of the tasks we require. The human brain, while having billions of neurons is still quite sparsely connected.

[ Parent | on HN ]