Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by bigyabai on Wednesday July 29, 2026 @02:21PM (#49101100)

You're kinda describing the MoE architecture; you can offload expert layers and stream them as-needed if the experts are small enough and the SSD is fast enough.

Dense LLMs typically perform better, but slow down much more than MoE models when you try offloading layers.

[ Parent | on HN ]

Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by avadodin on Wednesday July 29, 2026 @07:37PM (#49104458)

The issue is that the MoE knowledge is LLM "knowledge" and it still has a cost so overall it is lower quality/cost.

What he's envisioning is a dense 1B model that looks at the Python specification and your prompt and goes:

> Ah, I get it now! This is like Darmok and Jalad at Tanagra!

Or at least:

> Android UI development in Python? It's UNIX, I know this!

To an extent we do work like this sometimes but in general we rely on internalized knowledge so I don't know to what extent it is a viable strategy.

[ Parent | on HN ]