Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Stability AI Launches the First of Its StableLM Suite of Language Models. Parent comment
"standard attention has memory quadratic in sequence length, whereas FlashAttention has memory linear in sequence length."
I guess you have just reported how many times the layer will need to access the memory, not how much memory usage scales with sequence length.
There are no comments on this comment yet.
"standard attention has memory quadratic in sequence length, whereas FlashAttention has memory linear in sequence length."
I guess you have just reported how many times the layer will need to access the memory, not how much memory usage scales with sequence length.