Re:What happens if we remove 50 percent of Llama?
by sitkack on Monday December 02, 2024 @01:46PM
(#42299051)
> This basically shows over 50% of the neural net is gibberish! The reason being is that the objective function simply does not include it.
This is a mischaracterization of sparsity. Performance did drop, so the weights are not gibberish. Training vs pruning, you can't train into the final state, you can only prune there.
This is a mischaracterization of sparsity. Performance did drop, so the weights are not gibberish. Training vs pruning, you can't train into the final state, you can only prune there.