Re:RedPajama (Open LLaMA Replication) 3B and 7B models released
by readyplayeremma on Saturday May 06, 2023 @12:02AM
(#35837852)
What size model are you quantizing and comparing? The interesting thing about quantization, is how the larger the number of parameters, the less of a difference it makes to quantize the weights, even to an extreme degree when working with the largest parameter models. For small models is can be a disaster though.
[ Parent
| on HN ]
There are no comments on this comment yet.