News for hackers. Stuff that matters.

A comment on /story/43170155. Parent comment

Re: by Workaccount2 on Wednesday February 26, 2025 @11:28AM (#43185200)
Yeah, LLM capabilities are measured with fresh context windows, yet people want to use them with 50k, 100k, 500k tokens.

As you pack in more and more context the model's abilities really start to deteriorate.

The first 10k tokens are the juiciest, after that it just gets worse and worse.

[ Parent | on HN ]
Re: by eternityforest on Wednesday February 26, 2025 @11:35AM (#43185278)
Oh wow, I was thinking 500 tokens was way too much, since I've only ever done anything programmatic with tiny models on CPUs....
[ Parent | on HN ]