News for hackers. Stuff that matters.

A comment on Gemma 3 QAT Models: Bringing AI to Consumer GPUs. Parent comment

Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by a_e_k on Monday April 21, 2025 @01:02AM (#43748620)
I'm seeing ~38--42 tps on a 4090 in a fresh build of llama.cpp under Fedora 42 on my personal machine.

(-t 32 -ngl 100 -c 8192 -fa -ctk q8_0 -ctv q8_0 -m models/gemma-3-27b-it-qat-q4_0.gguf)

[ Parent | on HN ]

There are no comments on this comment yet.