News for hackers. Stuff that matters.
Re:QLoRA: Efficient Finetuning of Quantized LLMs by MacsHeadroom on Thursday May 25, 2023 @05:58PM (#36076825)
I'm not sure it could be for this complex quantization scheme. It's not a simple round to nearest quantization. Even if it is possible, it would require someone to develop that functionality. Currently it does load the entire unquantized model into RAM first.

There are no comments on this comment yet.