News for hackers. Stuff that matters.

A comment on Accelerating Gemma 4: faster inference with multi-token prediction drafters. Parent comment

Re:Accelerating Gemma 4: faster inference with multi-token p... by Urahandystar on Tuesday May 05, 2026 @02:33PM (#48026600)
True, but you have to add up the cumulative token output if your being fair. That alignment issue requires another set of input and output tokens to correct.
[ Parent | on HN ]
Re:Accelerating Gemma 4: faster inference with multi-token p... by MengerSponge on Tuesday May 05, 2026 @03:01PM (#48026959)
Does it? Or is this a centaur situation where a competent human can fix it in about two minutes?
[ Parent | on HN ]
Re:Accelerating Gemma 4: faster inference with multi-token p... by Schiendelman on Tuesday May 05, 2026 @08:39PM (#48030714)
Define competent. This is the difference between having a product manager able to prototype and having a product manager need to work with an engineer.
[ Parent | on HN ]
Re:Accelerating Gemma 4: faster inference with multi-token p... by mark_l_watson on Wednesday May 06, 2026 @09:10AM (#48035826)
Yes! Sometimes when models get something wrong with less widely used programming languages, I like to just cancel the current inference, fix something myself, then tell the harness/model that I fixed the current problem, and to move on.
[ Parent | on HN ]