News for hackers. Stuff that matters.
Re:Low-Rank Pruning of Llama2 by ibuildthings on Friday November 03, 2023 @04:50PM (#38134836)
Github repo should be visible now.

It is not distilling the model, it is reducing the model weights on the fly and uses LoRA for training/fine-tuning. After the training phase, we explain how to merge the LoRA weights with the pruned weights to achieve faster inference speed

There are no comments on this comment yet.