News for hackers. Stuff that matters.
Re:Outperforming LLMs with less training data and smaller mo... by stevenhuang on Thursday September 21, 2023 @11:12PM (#37607393)
That number is not quantized either. Quantize 175B param at 4 bits then it'd fit in ~120GB VRAM (one can fit 34B models at 4 bit quantization in a single RTX3090 24GB VRAM)

There are no comments on this comment yet.