Re:StreamingLLM: tiny tweak to KV LRU improves long conversa...
by joshspankit on Tuesday February 13, 2024 @08:58AM
(#39357638)
You’re right: A lot of the conversation can be condensed, especially if there are enough cues for the AI to arrive in the same “neuronal neighborhood” as the previous conversation.
[ Parent
| on HN ]
There are no comments on this comment yet.