News for hackers. Stuff that matters.

A comment on Anthropic - Introducing 100K Token Context Windows, Around 75,000 Words. Parent comment

Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by make3 on Thursday May 11, 2023 @03:48PM (#35907500)
Yes, caching the states of the sequence would make sense. An issue is that it's still more expensive to compute the new tokens even if you cache the states viewed so far
[ Parent | on HN ]

There are no comments on this comment yet.