Re:Show HN: I made a GPU VRAM calculator for transformer-bas...
by kkzz99 on Wednesday December 27, 2023 @06:36AM
(#38781036)
Usually, for efficiency, you use quantized models. Quantized models reduce the number of bits available for each parameter, saving space and reduce RAM usage.