News for hackers. Stuff that matters.
Re:Stability AI Launches the First of Its StableLM Suite of ... by GaggiX on Wednesday April 19, 2023 @06:45PM (#35634258)
https://github.com/HazyResearch/flash-attention#memory

"standard attention has memory quadratic in sequence length, whereas FlashAttention has memory linear in sequence length."

I guess you have just reported how many times the layer will need to access the memory, not how much memory usage scales with sequence length.

There are no comments on this comment yet.