News for hackers. Stuff that matters.
Re:Try Deep Think in the Gemini App by energy123 on Friday August 01, 2025 @01:32PM (#44759778)
No, you're talking about costs to user, which are oversimplifications of the costs that providers bear. One output token with a million input tokens is incredibly cheap for providers
Re:Try Deep Think in the Gemini App by danenania on Friday August 01, 2025 @02:06PM (#44760280)
> One output token with a million input tokens is incredibly cheap for providers

Source? Afaik this is incorrect.

Re:Try Deep Think in the Gemini App by danielbln on Friday August 01, 2025 @03:06PM (#44761031)
Chevk out any LLM API providers pricing. Output tokens are always significantly more expensive than input (which can also be cached).
Re:Try Deep Think in the Gemini App by danenania on Friday August 01, 2025 @03:40PM (#44761451)
Input tokens usually dominate output tokens by a lot more than 2x though. It’s often 10x or more input. It can even easily be 100x or more. Again in realistic workflows.

Caching does help the situation, but you always at least pay the initial cache write. And prompts need to be structured carefully to be cacheable. It’s not a free lunch.