News for hackers. Stuff that matters.
Re:Show HN: Open-source engine running Gemma 4 26B in 2 GB R... by avadodin on Wednesday July 29, 2026 @07:37PM (#49104458)
The issue is that the MoE knowledge is LLM "knowledge" and it still has a cost so overall it is lower quality/cost.

What he's envisioning is a dense 1B model that looks at the Python specification and your prompt and goes:

> Ah, I get it now! This is like Darmok and Jalad at Tanagra!

Or at least:

> Android UI development in Python? It's UNIX, I know this!

To an extent we do work like this sometimes but in general we rely on internalized knowledge so I don't know to what extent it is a viable strategy.

There are no comments on this comment yet.