News for hackers. Stuff that matters.

A comment on Show HN: I made a GPU VRAM calculator for transformer-based models.

Re:Show HN: I made a GPU VRAM calculator for transformer-bas... by ilaksh on Tuesday December 26, 2023 @04:53PM (#38776493)
Does this have an option for quantization levels? Don't think I saw it.
[ Parent | on HN ]
Re:Show HN: I made a GPU VRAM calculator for transformer-bas... by furiousteabag on Tuesday December 26, 2023 @08:21PM (#38778020)
There is no option to select quantized version yet. Will work on that!
[ Parent | on HN ]
Re:Show HN: I made a GPU VRAM calculator for transformer-bas... by ComputerGuru on Tuesday December 26, 2023 @05:22PM (#38776743)
I second the request for quantization, eg for exl2.
[ Parent | on HN ]