News for hackers. Stuff that matters.
Re:Microsoft Phi-2 model changes licence to MIT by regularfry on Saturday January 06, 2024 @07:13PM (#38896980)
Key-value cache in the attention layers. There was a paper a little while back about how maintaining the first N tokens across an extended context helped an LLM keep sane for longer, and it turns out you can replicate it with the right CLI arguments to llama.cpp.

There are no comments on this comment yet.