Re:Llama.cpp speculative sampling: 2x faster inference for l...
by xchip on Tuesday September 05, 2023 @07:43AM
(#37390415)
Small caveat: this is only true for generating text with simple grammar, like code. For human text this doesn't work so well.
[ Parent
| on HN ]