Re:Mammals dream about the world they are about to experienc...
by stared on Tuesday July 27, 2021 @05:23AM
(#27970368)
I think it is less relevant to Winning Ticket Hypothesis (in my opinion, the most important paper on deep learning theory in the last 5 years), but about architectures that work to some extend with random weights.
> Inspired by precocial species in biology, we set out to search for neural net architectures that can already (sort of) perform various tasks even when they use random weight values.
See "Weight Agnostic Neural Networks" (https://weightagnostic.github.io/):
> Inspired by precocial species in biology, we set out to search for neural net architectures that can already (sort of) perform various tasks even when they use random weight values.