Qwen2.5-1M: Deploy your own Qwen with context length up to 1M tokens

Re:Qwen2.5-1M: Deploy your own Qwen with context length up t... by cyanydeez on Monday January 27, 2025 @04:37PM (#42845930)

That's what these char models are already doing.

Most chat is just a long running prompt. LLMs have zero actual memory. You just keep feeding it history.

Maybe I misunderstood what you're saying but what you're describing is some kind of 2nd model that condenses that history and that gets fed; this has been done.

Really, what you probably need is another model managing the heap and the stack of the history and brining forward the current context.

But that's easy to say because we are humans.

[ Parent | on HN ]

Re:Qwen2.5-1M: Deploy your own Qwen with context length up t... by NiloCK on Tuesday January 28, 2025 @07:49AM (#42851711)

To clarify:

I didn't 'fix' the problem by re-inserting the package.json. I just gave a reminder that it that it was already in context.

"I could confirm this root cause if I could see the contents of package.json".

"You can see it."

"Whoops, yes. Package x needs to bump from n to n+1."

The point being, even info inside the current context can actively be overlooked unless it is specifically referenced.

[ Parent | on HN ]