Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel...
by liuliu on Sunday April 21, 2024 @08:58PM
(#40110612)
I might be naive comes to classic ML / NLP: how do you keep the prefix table for N = 5? Naively, that look-up table would be 100k^5 (assuming 100k vocabulary size). Is it very sparse? How large that usually is?
[ Parent
| on HN ]
There are no comments on this comment yet.