Re:Accelerating Gemma 4: faster inference with multi-token p...
by 332451b on Tuesday May 05, 2026 @07:25PM
(#48030113)
I'm curious how you got the energy consumption for 8 hours of use.
I would imagine there's a huge spread there too. Depending on hardware, size of the model, requests being batched and queued rather than occupying a whole GPU or TPU, or if you are running an agent loop for 8 hours straight or spend most of the time reviewing results.
I would imagine there's a huge spread there too. Depending on hardware, size of the model, requests being batched and queued rather than occupying a whole GPU or TPU, or if you are running an agent loop for 8 hours straight or spend most of the time reviewing results.