News for hackers. Stuff that matters.

A comment on Qwen2.5 Turbo extends context length to 1M tokens. Parent comment

Re:Qwen2.5 Turbo extends context length to 1M tokens by nisarg2 on Monday November 18, 2024 @01:14PM (#42175171)
At the end of the response they forget everything. They need to be fed the entire text for them to know anything about it the next time. That is not surpassing even feline intelligence.
[ Parent | on HN ]
Re:Qwen2.5 Turbo extends context length to 1M tokens by andai on Monday November 18, 2024 @01:17PM (#42175217)
A genius can have anterograde amnesia and still be a genius.
[ Parent | on HN ]
Re:Qwen2.5 Turbo extends context length to 1M tokens by anon291 on Monday November 18, 2024 @01:23PM (#42175305)
If we did to cats what he did to GPT models that would be animal abuse.

That is to say, if we want to extend this analogy, the model is 'killed' after each round. This is hardly a criticism of the underlying technology.

Going back to feeding the entire input. That is not really true. There are a dozen ways to not do that these day.

[ Parent | on HN ]