Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs
by halflings on Sunday April 20, 2025 @09:28AM
(#43743634)
That's what the chart says yes. 14.1GB VRAM usage for the 27B model.
[ Parent
| on HN ]
Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs
by erichocean on Sunday April 20, 2025 @09:34AM
(#43743678)
That's the VRAM required just to load the model weights.
To actually use a model, you need a context window. Realistically, you'll want a 20GB GPU or larger, depending on how many tokens you need.