Re:QuIP#: 2-bit Quantization for LLMs
by garrisonhh on Saturday December 09, 2023 @09:21AM
(#38582116)
Yes but no. The actual values represented by the quantized bits don't use a representation akin to IEEE floating point, but they are able to act like floating point values due to mathematical transformations during propagation. The floating point values a quantized value corresponds to are chosen using some kind of precomputation depending on the quantization method
[ Parent
| on HN ]
There are no comments on this comment yet.