Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by pwython on Wednesday July 29, 2026 @09:31PM (#49105161)

Ran this on a 64 GB M4 Max MacBook. I figured having Gemma available with a small footprint would be a nice setup. No more unloading models when I need more RAM for work? Hell yea.

Got 48 tok/s decode at 1.9 GB RSS (2.4 GB peak), faster than the 24 GB M5 Pro mentioned in the benchmarks. The ~2.0 GB/s SSD number quoted for M4 is the base chip. This M4 Max does ~7 GB/s.

Page cache seems to be why it beats the M5 Pro. With 64 GB the whole 12 GB packed_experts set stays resident, and iostat shows only ~1.6 GB per run actually reaching disk, against the ~79 GB that 98 fully cold tokens would need.

I then tested with DaVinci Resolve open and under load (playback): 42.6 tok/s. Also held 38 GB of incompressible memory to squeeze the page cache: 41.8. At 48 GB it ranged 32 to 41.5. Degrades gradually rather than a cliff. It's a beautiful thing.

[ Parent | on HN ]

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by fouc on Wednesday July 29, 2026 @10:00PM (#49105337)

M4 Max is typically better than M5 Pro for inference IIRC.

[ Parent | on HN ]

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by harrouet on Thursday July 30, 2026 @04:06AM (#49107218)

It depends on what you are looking at.

Time to 1st token is faster on the M5 because of HW accelerators helping the prompt interpretation (and it is CPU-bound).

Token generation after that is GPU-bound and will profit from the higher bandwidth of the M4 Max.

[ Parent | on HN ]

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by sznio on Thursday July 30, 2026 @03:54AM (#49107146)

at that much ram you can just load it outright without tricks. it will be much faster even if it ends up swapping.

[ Parent | on HN ]

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by gitpusher42 on Thursday July 30, 2026 @04:19AM (#49107305)

Thank you very much for sharing! Great results and useful info!

[ Parent | on HN ]

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by pitchlatte on Thursday July 30, 2026 @05:35AM (#49107762)

playback in Resolve would probably just use hardware decoding and barely hit your CPU or GPU. RAM usage would also not be much.

[ Parent | on HN ]

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by anon373839 on Thursday July 30, 2026 @08:20AM (#49109018)

Nice! Can you mention what kind of prefill numbers you’re seeing?

[ Parent | on HN ]