News for hackers. Stuff that matters.

A comment on Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac. Parent comment

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by gitpusher42 on Wednesday July 29, 2026 @12:00PM (#49099223)
The process stays at around 2 GB with 16 slots and a 4K context on both the M5 and M2. But yeah, Apple might be doing some magic under the hood
[ Parent | on HN ]
Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by petu on Wednesday July 29, 2026 @12:26PM (#49099575)
Unused RAM is wasted RAM. So not really Apple magic, about every OS uses "free" memory as disk cache.

Try to leave only a gigabyte or two free, speed likely would drop dramatically.

Edit: or do some calculation / logging of experts read speed, to see if it's faster than SSD spec.

[ Parent | on HN ]
Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by gitpusher42 on Thursday July 30, 2026 @08:22AM (#49109034)
yeah, looks like a page cache matters a lot I tested on mine m5 pro with 8gb memory pressure, got 27t/s instead of 35t/s Someone tested on m4 max. In regular state it was 48tok/s, but 32-42 with memory pressure
[ Parent | on HN ]