News for hackers. Stuff that matters.
Re:Gemini 3 Flash: frontier intelligence built for speed by danpalmer on Wednesday December 17, 2025 @11:03PM (#46308803)
I thought it was generally accepted that inference was faster on TPUs. This was one of my takeaways from the LLM scaling book: https://jax-ml.github.io/scaling-book/ – TPUs just do less work, and data needs to move around less for the same amount of processing compared to GPUs. This would lead to lower latency as far as I understand it.

The citation link you provided takes me to a sales form, not an FAQ, so I can't see any further detail there.

> Both Cerebras and Grok have custom AI-processing hardware (not CPUs).

I'm aware of Cerebras' custom hardware. I agree with the other commenter here that I haven't heard of Grok having any. My point about knowledge grounding was simply that Grok may be achieving its latency with guardrail/knowledge/safety trade-offs instead of custom hardware.

Re:Gemini 3 Flash: frontier intelligence built for speed by nl on Thursday December 18, 2025 @12:47AM (#46309285)
Sorry I meant Groq custom hardware, not Grok!

I don't see any latency comparisons in the link

Re:Gemini 3 Flash: frontier intelligence built for speed by danpalmer on Thursday December 18, 2025 @02:03AM (#46309686)
The link is just to the book, the details are scattered throughout. That said the page on GPUs specifically speaks to some of the hardware differences and how TPUs are more efficient for inference, and some of the differences that would lead to lower latency.

https://jax-ml.github.io/scaling-book/gpus/#gpus-vs-tpus-at-...

Re: Groq, that's a good point, I had forgotten about them. You're right they too are doing a TPU-style systolic array processor for lower latency.