Re:Only Train Once: A One-Shot Neural Network Training and P...
by anonymousDan on Saturday July 17, 2021 @06:55AM
(#27864938)
Thanks. Can you give an intuition as to the difference between 'structured' and 'unstructured' pruning? What are the pros and cons of each?
[ Parent
| on HN ]
Re:Only Train Once: A One-Shot Neural Network Training and P...
by ndronen on Wednesday July 28, 2021 @01:20PM
(#27986347)
In unstructured pruning, individual weights of a weight tensor are pruned with no constraints on their position in the tensor. In structured pruning, there are constraints on which weights are pruned. Think of pruning an entire output channels of a convolutional layer vs pruning arbitrary weights of the same tensor. Unstructured pruning allows for a greater percentage of the weights to be pruned, but unstructured pruning doesn't speed up a network on hardware accelerators, which favor dense matrix multiplications.