News for hackers. Stuff that matters.
Re:StreamingLLM: tiny tweak to KV LRU improves long conversa... by joshspankit on Tuesday February 13, 2024 @08:58AM (#39357638)
You’re right: A lot of the conversation can be condensed, especially if there are enough cues for the AI to arrive in the same “neuronal neighborhood” as the previous conversation.

There are no comments on this comment yet.