Re:Batch Mode in the Gemini API: Process More for Less
by graeme on Friday July 11, 2025 @12:48AM
(#44528499)
Seems more like electricity pricing, which has peak and offpeak pricing for most business customers.
To handle peak daily load you need capacity that goes unused in offpeak hours.
Re:Batch Mode in the Gemini API: Process More for Less
by reasonableklout on Friday July 11, 2025 @12:48AM
(#44528501)
Why do you think that this means "idle GPU" rather than a company recognizing a growing need and allocating resources toward it?
It's cheaper because it's a different market with different needs which can be served by systems optimizing for throughput instead latency. Feels like you're looking for something that's not there.
In a way this is saying that there are some GPUs just sitting around so they would rather get 50% than nothing for their use.