Re:26× Faster Inference with Layer-Condensed KV Cache for La...
by 0cf8612b2e1e on Monday May 20, 2024 @04:15PM
(#40419771)
Even if it is “only” the 40% lower end, that is a gargantuan savings. So many groups are compute constrained, every bit helps.
[ Parent
| on HN ]