News for hackers. Stuff that matters.
Re:Gemma 4 12B: A unified, encoder-free multimodal model by hedgehog on Wednesday June 03, 2026 @11:09PM (#48393228)
Same chip, with a 6 bit 35B and 8 bit KV cache I see about 500 prefill and 55 decode at 30k into the context window. MiniMax seemed a bit lower token rate but much, much less prone to 40k tokens of monologue before generating an answer. A pattern I like is to use a smaller model to do most execution and then a larger model to review transcripts and output and do any fixups and tooling improvements (this is all batch jobs so all I care about is overall throughput).

There are no comments on this comment yet.