Deep learning model compression methods

Re:Deep learning model compression methods by posharma on Monday April 05, 2021 @05:47AM (#26697045)

I’m surprised there’s no mention of sparse formats to store the weights of the model.

[ Parent | on HN ]

Re:Deep learning model compression methods by antixk on Monday April 05, 2021 @06:02AM (#26697114)

Becuase unless you have a certain (high) level of sparsity, sparse formats are infact ineffective in storing. There are cases where sparse formats take more memory than storing dense tensors.

[ Parent | on HN ]

Re:Deep learning model compression methods by posharma on Monday April 05, 2021 @06:24AM (#26697224)

Sure. It’s still one of the methods to compress models. You might be surprised how often the weights are really quite sparse.

[ Parent | on HN ]