News for hackers. Stuff that matters.
Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel... by kristjansson on Sunday April 21, 2024 @04:10PM (#40108806)
This is speculative decoding with a n-gram Markov chain instead of a weaker transformer model in the “speculating” position.
Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel... by huevosabio on Sunday April 21, 2024 @06:54PM (#40109937)
Thanks, this is a great, succinct way of summarizing the paper.