Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel...
by tripplyons on Sunday April 21, 2024 @09:11PM
(#40110679)
They use a separate ngram model to generate the proposed sequence instead of extra heads on top of the main model. The process of verifying the proposed sequence appears to be the same.
https://arxiv.org/abs/1811.03115