News for hackers. Stuff that matters.

A comment on Gemini Flash. Parent comment

Re:Gemini Flash by mupuff1234 on Tuesday May 14, 2024 @03:49PM (#40359368)
I think that's a bit like asking why would someone need a 1gb Gmail when 50mb yahoo account is clearly enough.

It means you can dump context without thinking about it twice and without needing to hack some solutions to deal with context overflow etc.

And given that most use cases most likely deal with text and not multimodal the advantage seems pretty clear imo.

[ Parent | on HN ]
Re:Gemini Flash by tedsanders on Tuesday May 14, 2024 @04:21PM (#40359718)
Long context is a little bit different than extra email storage. Having 1 gb of storage instead of 50 mb has essentially no downside to the user experience.

But submitting 1M input tokens instead of 100k input tokens:

- Causes your costs to go up ~10x

- Causes your latency to go up ~10x (or between 1x and 10x)

- Can result in worse answers (especially if the model gets distracted by irrelevant info)

So longer context is great, yes, but it's not a no-brainer like more email storage. It brings costs. And whether those costs are worth it depends on what you're doing.

[ Parent | on HN ]