Re:QLoRA: Efficient Finetuning of Quantized LLMs
by MacsHeadroom on Thursday May 25, 2023 @05:58PM
(#36076825)
I'm not sure it could be for this complex quantization scheme. It's not a simple round to nearest quantization. Even if it is possible, it would require someone to develop that functionality. Currently it does load the entire unquantized model into RAM first.
[ Parent
| on HN ]
There are no comments on this comment yet.