News for hackers. Stuff that matters.

A comment on Qwen2.5-1M: Deploy your own Qwen with context length up to 1M tokens. Parent comment

Re:Qwen2.5-1M: Deploy your own Qwen with context length up t... by badlogic on Monday January 27, 2025 @09:53AM (#42841713)
I concur. In my work (analysing news show transcripts and descriptions), I work with about 250k input tokens max. Tasks include:

- Summarize topics (with references to shows) - Find quotes specific to a topic (again with references)

Anything above 32k tokens fails to have acceptable recall, across GPT-4o, Sonnet, and Google's Gemini Flash 1.5 and 2.0.

I suppose it kind of makes sense, given how large context windows are implemented via things like sparse attention etc.

[ Parent | on HN ]
Re:Qwen2.5-1M: Deploy your own Qwen with context length up t... by kgeist on Monday January 27, 2025 @02:36PM (#42844773)
What could be the reason? Do they selectively skip tokens to make it appear they support the full context?
[ Parent | on HN ]