Re:Writing Speed-of-Light Flash Attention for 5090 in CUDA C++
by saagarjha on Saturday August 23, 2025 @10:31PM
(#45000802)
There's a 2x performance hit from the weird restriction on fp32 accumulation, plus the fact that 5090 has "fake" Blackwell (no tcgen05) which limits the size and throughput of matrix multiplication through the tensor cores.
[ Parent
| on HN ]
There are no comments on this comment yet.