News for hackers. Stuff that matters.

A comment on FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-Precision.

Re:FlashAttention-3: Fast and Accurate Attention with Asynch... by ex3ndr on Thursday July 11, 2024 @04:39PM (#40940425)
I am wondering why flash attention is like 5x slower with variable masking than without it? Lack of good masking support almost zeros out the optimizations
[ Parent | on HN ]
Re:FlashAttention-3: Fast and Accurate Attention with Asynch... by chillee on Friday July 12, 2024 @02:38AM (#40943181)
Where are you seeing these benchmarks?
[ Parent | on HN ]