News for hackers. Stuff that matters.

A comment on Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac. Parent comment

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by gitpusher42 on Wednesday July 29, 2026 @11:38AM (#49098926)
It depends on the use case.

I measured this exact model with a 4k context on the mlx engine. It runs at 75 tok/s on my M5 Mac Pro and using 14 GB of RAM. For my engine the same model uses 2 GB of RAM and produces 31–35 tok/s.

The project is still experimental so performance may vary as it continues to improve. If you want to save around 12 GB of RAM for other tasks and you are ok with 35 tok/s (afaik it is roughly comparable to ChatGPT’s speed for basic responses) my engine may be a good fit.

If you need maximum speed and flexibility just use MLX

[ Parent | on HN ]
Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by anentropic on Thursday July 30, 2026 @07:09AM (#49108357)
can I vary the context length depending on RAM available?
[ Parent | on HN ]
Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by gitpusher42 on Thursday July 30, 2026 @07:24AM (#49108495)
Yeah, sure! You can select different options in the app settings at the right panel, it shows how much memory it will use

For CLI and Server, use --max-context

[ Parent | on HN ]