News for hackers. Stuff that matters.
Re:Gemini 3 Flash: frontier intelligence built for speed by mips_avatar on Wednesday December 17, 2025 @07:02PM (#46307325)
It's weird they don't document this stuff. Like understanding things like tool call latency and time to first token is extremely important in application development.
Re:Gemini 3 Flash: frontier intelligence built for speed by eru on Wednesday December 17, 2025 @09:18PM (#46308249)
Humans often answer with fluff like "That's a good question, thanks for asking that, [fluff, fluff, fluff]" to give themselves more breathing room until the first 'token' of their real answer. I wonder if any LLM are doing stuff like that for latency hiding?
Re:Gemini 3 Flash: frontier intelligence built for speed by strangegecko on Wednesday December 17, 2025 @11:40PM (#46308946)
Do humans really do that often?

Coming up with all that fluff would keep my brain busy, meaning there's actually no additional breathing room for thinking about an answer.

Re:Gemini 3 Flash: frontier intelligence built for speed by eru on Thursday December 18, 2025 @12:27AM (#46309161)
People who professionally answer questions do that, yes. Eg politicians or press secretaries for companies, or even just your professor taking questions after a talk.

> Coming up with all that fluff would keep my brain busy, meaning there's actually no additional breathing room for thinking about an answer.

It gets a lot easier with practice: your brain caches a few of the typical fluff routines.

Re:Gemini 3 Flash: frontier intelligence built for speed by mips_avatar on Wednesday December 17, 2025 @10:27PM (#46308614)
I don't think the models are doing this, time to first token is more of a hardware thing. But people writing agents are definitely doing this, particularly in voice it's worth it to use a smaller local llm to handle the acknowledgment before handing it off.