Re:Gemini 3 Flash: frontier intelligence built for speed
by mips_avatar on Wednesday December 17, 2025 @07:02PM
(#46307325)
It's weird they don't document this stuff. Like understanding things like tool call latency and time to first token is extremely important in application development.
[ Parent
| on HN ]
Re:Gemini 3 Flash: frontier intelligence built for speed
by eru on Wednesday December 17, 2025 @09:18PM
(#46308249)
Humans often answer with fluff like "That's a good question, thanks for asking that, [fluff, fluff, fluff]" to give themselves more breathing room until the first 'token' of their real answer. I wonder if any LLM are doing stuff like that for latency hiding?
Re:Gemini 3 Flash: frontier intelligence built for speed
by strangegecko on Wednesday December 17, 2025 @11:40PM
(#46308946)
Do humans really do that often?
Coming up with all that fluff would keep my brain busy, meaning there's actually no additional breathing room for thinking about an answer.
Re:Gemini 3 Flash: frontier intelligence built for speed
by eru on Thursday December 18, 2025 @12:27AM
(#46309161)
People who professionally answer questions do that, yes. Eg politicians or press secretaries for companies, or even just your professor taking questions after a talk.
> Coming up with all that fluff would keep my brain busy, meaning there's actually no additional breathing room for thinking about an answer.
It gets a lot easier with practice: your brain caches a few of the typical fluff routines.
Re:Gemini 3 Flash: frontier intelligence built for speed
by mips_avatar on Wednesday December 17, 2025 @10:27PM
(#46308614)
I don't think the models are doing this, time to first token is more of a hardware thing. But people writing agents are definitely doing this, particularly in voice it's worth it to use a smaller local llm to handle the acknowledgment before handing it off.