QuIP#: 2-bit Quantization for LLMs

Re:QuIP#: 2-bit Quantization for LLMs by rolisz on Saturday December 09, 2023 @04:00AM (#38580044)

If we get 1-bit quantization, wouldn't it be basically a bunch of nested if's and else's?

[ Parent | on HN ]

Re:QuIP#: 2-bit Quantization for LLMs by compinter on Saturday December 09, 2023 @06:20AM (#38580936)

So, just like the rest of my code then

[ Parent | on HN ]

Re:QuIP#: 2-bit Quantization for LLMs by hskalin on Saturday December 09, 2023 @08:48AM (#38581899)

So a decision tree? They are good for some tasks I reckon

[ Parent | on HN ]

Re:QuIP#: 2-bit Quantization for LLMs by patrakov on Sunday December 10, 2023 @11:34AM (#38592716)

No. The catch is that what's quantized here is not individual weights but groups of them.

[ Parent | on HN ]