Re:26× Faster Inference with Layer-Condensed KV Cache for La...
by joaquincabezas on Monday May 20, 2024 @03:50PM
(#40419476)
LLM inference optimization has been key for the OpenAI GPT-4o presentation (2x faster, 50% cheaper) and its driving lots of industry research because it’s direct cost savings, but it’s refreshing to see so many techniques published as papers (i.e from Stanford, Berkeley…)
[ Parent
| on HN ]
There are no comments on this comment yet.