News for hackers. Stuff that matters.
Re:QuIP#: 2-bit Quantization for LLMs by garrisonhh on Saturday December 09, 2023 @09:21AM (#38582116)
Yes but no. The actual values represented by the quantized bits don't use a representation akin to IEEE floating point, but they are able to act like floating point values due to mathematical transformations during propagation. The floating point values a quantized value corresponds to are chosen using some kind of precomputation depending on the quantization method

There are no comments on this comment yet.