Re:Deep image prior 'learns' on just one image
by amelius on Friday December 01, 2017 @08:35AM
(#15823680)
Another way to think of it:
Convnets are a method to speed up learning by sharing filter-coefficients. In other words, one could just as well train the network by replacing the convnets by fully-connected layers, except that of course training would take a lot longer.
If one views convnets as purely a computational optimization, then the approach of the paper can be considered a "hack" that only works because of this optimization.
> This makes me wonder whether and to what degree the structure of the brain's connectome might be a necessary prior for AGI.
Convnets have been invented to speed up learning (reusing parameters). The human brain (afaik) doesn't contain such speedup. Instead it relies on massive parallellism of neurons, and thus trains all "filters" separately instead of reusing them. Therefore, I suppose this approach would not apply to humans.