Accelerating Gemma 4: faster inference with multi-token prediction drafters

Re:Accelerating Gemma 4: faster inference with multi-token p... by asdfasgasdgasdg on Tuesday May 05, 2026 @08:34PM (#48030676)

I'm using it in antigravity, and fint it quite good. I have not managed to run out of usage on Flash. You can run Pro out of quota almost instantly, they really don't want you to use it if you're not paying $200 a month.

I do not use super broad prompts, though. None of this "build me a webapp" stuff. It's more like, "adjust this part of this class to do Y instead of X."

[ Parent | on HN ]

Re:Accelerating Gemma 4: faster inference with multi-token p... by qingcharles on Tuesday May 05, 2026 @10:36PM (#48031467)

Also bonus: using it in Antigravity you can burn through all the Opus credit Google give you first to do all the planning and then switch it to Gemini 3.1 Pro to do the grunt work.

[ Parent | on HN ]

Re:Accelerating Gemma 4: faster inference with multi-token p... by xnx on Tuesday May 05, 2026 @11:28PM (#48031803)

Have you compared Opus and Gemini to see if Gemini is any worse at planning than Opus?

[ Parent | on HN ]

Re:Accelerating Gemma 4: faster inference with multi-token p... by qingcharles on Wednesday May 06, 2026 @01:14AM (#48032462)

Yes, Gemini 3.1 Pro (High) is still inferior to Opus 4.6 (Thinking) that Google are offering, for planning. It just doesn't think things through as thoroughly as Opus. I'll use it when I've burned up all my Opus tokens and I still have planning I want to do, but I'll read the plan very carefully, whereas with Opus I'll only give it a cursory scan through.

[ Parent | on HN ]

Re:Accelerating Gemma 4: faster inference with multi-token p... by xnx on Wednesday May 06, 2026 @01:17AM (#48032488)

Good data point. I would venture 90+% of Claude users have dismissed Gemini without every trying it.

[ Parent | on HN ]

Re:Accelerating Gemma 4: faster inference with multi-token p... by rjh29 on Wednesday May 06, 2026 @02:47AM (#48033069)

If you use the Pro model, it can handle fairly broad prompts. Flash is very basic (no thinking)

[ Parent | on HN ]

Re:Accelerating Gemma 4: faster inference with multi-token p... by asdfasgasdgasdg on Wednesday May 06, 2026 @09:49AM (#48036239)

Sure, but with the $15/mo plan you run out of pro so fast that I prefer not to rely on it. I'll do broader prompts in two years when the cheap models are as smart as the frontier models are today.

[ Parent | on HN ]