I ran Gemma 4 as a local model in Codex CLI

Re:I ran Gemma 4 as a local model in Codex CLI by blackmanta on Sunday April 12, 2026 @09:27PM (#47746477)

With a nvidia spark or 128gb+ memory machine, you can get a good speed up on the 31B model if you use the 26B MoE as a draft model. It uses more memory but I’ve seen acceptance rate at around 70%+ using Q8 on both models

[ Parent | on HN ]

Re:I ran Gemma 4 as a local model in Codex CLI by foobar10000 on Sunday April 12, 2026 @10:11PM (#47746744)

1 token ahead or 2?

It's interesting - imo we'll soon have draft models specifically post-trained for denser, more complicated models. Wouldn't be surprised if diffusion models made a comeback for this - they can draft many tokens at once, and learning curves seem to top out at 90+% match for auto-regressive ones so quite interesting..

[ Parent | on HN ]

Re:I ran Gemma 4 as a local model in Codex CLI by electroglyph on Monday April 13, 2026 @06:10AM (#47749978)

flow matching is making some strides right now, too

[ Parent | on HN ]