Re:Neural Network Quine
by visarga on Wednesday March 21, 2018 @01:46AM
(#16635781)
> What’s the significance of this paper?
We're progressing from static neural nets (like the old TensorFlow) to dynamic nets (like PyTorch) and even dynamically generated on-the-fly neural nets (hypernets) which is great for creating architectures that better fit the problem, case by case.
This paper shows another way to generate a net from a net, but this time with self replication. Self replication could be useful for evolutionary optimisation, and, as they say in the paper, could become a rudimentary form of introspection. The idea of a "self-replication loss term" is very exciting, it's rare that we see such innovation in loss functions.
We're progressing from static neural nets (like the old TensorFlow) to dynamic nets (like PyTorch) and even dynamically generated on-the-fly neural nets (hypernets) which is great for creating architectures that better fit the problem, case by case.
This paper shows another way to generate a net from a net, but this time with self replication. Self replication could be useful for evolutionary optimisation, and, as they say in the paper, could become a rudimentary form of introspection. The idea of a "self-replication loss term" is very exciting, it's rare that we see such innovation in loss functions.