News for hackers. Stuff that matters.

A comment on Gemma 4 QAT models: Optimizing compression for mobile and laptop efficiency. Parent comment

Re:Gemma 4 QAT models: Optimizing compression for mobile and... by steno132 on Friday June 05, 2026 @05:09PM (#48418331)
But practically AI inference requires substantial local computing resources. It's not some web app, it's a order of magnitude more compute needed
[ Parent | on HN ]
Re:Gemma 4 QAT models: Optimizing compression for mobile and... by Zambyte on Friday June 05, 2026 @05:19PM (#48418473)
Hopefully now you understand why people want smaller models.
[ Parent | on HN ]
Re:Gemma 4 QAT models: Optimizing compression for mobile and... by satvikpendem on Friday June 05, 2026 @05:25PM (#48418562)
Not really, I run a production service on a basic server using these Gemma models, the server is weaker than my MacBook. Most people's laptops and even phones actually can run local models, most simply don't know how. Run Unsloth Studio and you'll see how easy it is.

As the sibling says this is why people want smaller but still performant models.

[ Parent | on HN ]