News for hackers. Stuff that matters.

A comment on Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac.

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by WithinReason on Wednesday July 29, 2026 @11:36AM (#49098888)
Nice job implementing expert caching!
[ Parent | on HN ]
Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by gitpusher42 on Wednesday July 29, 2026 @11:45AM (#49099012)
Thank you! Under good conditions it achieves approx a 67% cache hit rate with 16 expert slots
[ Parent | on HN ]
Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by WithinReason on Wednesday July 29, 2026 @01:33PM (#49100504)
That's great, now I wonder how cache hit rate scales for larger models. Do you have any plans trying Qwen 3.6 or larger?
[ Parent | on HN ]
Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by gitpusher42 on Wednesday July 29, 2026 @07:52PM (#49104560)
Check for colibri, dwarf star and flash-moe. they do similar things with bigger models

https://github.com/JustVugg/colibri https://github.com/antirez/ds4 https://github.com/danveloper/flash-moe

[ Parent | on HN ]