Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel...
by tripplyons on Sunday April 21, 2024 @09:11PM
(#40110679)
They use a separate ngram model to generate the proposed sequence instead of extra heads on top of the main model. The process of verifying the proposed sequence appears to be the same.
[ Parent
| on HN ]
There are no comments on this comment yet.