Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel...
by kristjansson on Sunday April 21, 2024 @04:10PM
(#40108806)
This is speculative decoding with a n-gram Markov chain instead of a weaker transformer model in the “speculating” position.
[ Parent
| on HN ]
Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel...
by huevosabio on Sunday April 21, 2024 @06:54PM
(#40109937)
Thanks, this is a great, succinct way of summarizing the paper.