News for hackers. Stuff that matters.

A comment on Gemma 3 QAT Models: Bringing AI to Consumer GPUs. Parent comment

Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by jychang on Monday April 21, 2025 @07:37AM (#43750755)
That's pretty terrible. I'm getting 18tok/sec Gemma 3 27b QAT on a M1 Max 32gb macbook.
[ Parent | on HN ]
Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by pantulis on Monday April 21, 2025 @07:45AM (#43750823)
Yeah, I know. Not sure if this due to something in LLM Studio or whatever.
[ Parent | on HN ]
Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by pantulis on Saturday April 26, 2025 @01:19PM (#43805417)
I was running the mac in low power mode!!! Getting 24tps now.
[ Parent | on HN ]