Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs
by jychang on Monday April 21, 2025 @03:47AM
(#43749341)
Those are the same quant, but this is a good example of why you shouldn't use ollama. Either directly use llama.cpp, or use something like LM Studio if you want something with a GUI/easier user experience.
The Gemma 3 17b QAT GGUF should be taking up ~15gb, not 22gb.
The Gemma 3 17b QAT GGUF should be taking up ~15gb, not 22gb.