News for hackers. Stuff that matters.
Re:Gradients Are Not All You Need by kyllo on Sunday April 23, 2023 @08:32PM (#35682274)
Are you familiar with the edge popup algorithm introduced in "What's Hidden in a Randomly Weighted Neural Network?" https://arxiv.org/abs/1911.13299v2

Seems relevant to what you're working on. It starts with a randomly initialized, overparameterized neural net, but instead of gradient descent backpropagation, it learns by deleting connection edges.

Re:Gradients Are Not All You Need by smonn_ on Monday April 24, 2023 @03:54AM (#35684805)
I haven't read it, thanks a lot! I'm probably going to use it in an essay I'm writing about the topic.