Re:Lossless Acceleration of LLM via Adaptive N-Gram Parallel...
by SonOfLilit on Monday April 22, 2024 @12:00PM
(#40115613)
As I said, the attention step is O(n^2) per token sequentially and O(n^2) for the entire sequence when calculating the entire sequence in parallel, where n is the length of the sequence.
[ Parent
| on HN ]
There are no comments on this comment yet.