Re:Outperforming LLMs with less training data and smaller mo...
by stevenhuang on Thursday September 21, 2023 @11:12PM
(#37607393)
That number is not quantized either. Quantize 175B param at 4 bits then it'd fit in ~120GB VRAM (one can fit 34B models at 4 bit quantization in a single RTX3090 24GB VRAM)
[ Parent
| on HN ]
There are no comments on this comment yet.