Re:Direct initialization of transformers using larger pretra...
by mirekrusin on Friday December 22, 2023 @05:09PM
(#38739242)
Instead of pruning least contributing neurons wouldn't it be better to ie. nominate next lowest contributing neuron or most similar one to merge them into one or redistribute its impact onto others?
[ Parent
| on HN ]
There are no comments on this comment yet.