Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel...
by johntb86 on Sunday April 21, 2024 @03:48PM
(#40108632)
They mention previous work on speculative decoding using similar techniques, but "ANPD dynamically generates draft outputs
via an adaptive N-gram module using real-time
statistics, after which the drafts are verified by the
LLM. This characteristic is exactly the difference
between ANPD and the previous speculative decoding methods."
[ Parent
| on HN ]
There are no comments on this comment yet.