10b5-1

I'm not assuming that, I'm giving the model more options and letting it decide what is functionally important/non-spurious. It might take it longer, but I don't assume that.

Looks like these researchers are trying to make a network more adaptive, I think that deleting nodes would only make them worse at the current task they're being trained on as well as worse on the tasks they're being adapted to.

You could train a model using neurogenesis to increase its accuracy, and then use distillation to train a smaller network to comparable accuracy.

But these are two very different, but complementary, problems.