Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on 26× Faster Inference with Layer-Condensed KV Cache for Large Language Models. Parent comment
It’s not hard to find ways to speed up transformers if you’re willing to give up quality.
You could argue some tradeoffs are worth it and that’s true sometimes but I don’t see that they’ve made the case for it here.
There are no comments on this comment yet.
It’s not hard to find ways to speed up transformers if you’re willing to give up quality.
You could argue some tradeoffs are worth it and that’s true sometimes but I don’t see that they’ve made the case for it here.