Writing Speed-of-Light Flash Attention for 5090 in CUDA C++

Re:Writing Speed-of-Light Flash Attention for 5090 in CUDA C++ by steinvakt2 on Saturday August 23, 2025 @02:24PM (#44997985)

Isn't 5090 FE (roughly 2500 USD in my country) pretty good FLOP value? 32 GB VRAM (and flash attention pushes it even faster compared to apple/mps relatively cheap "vram")

[ Parent | on HN ]

Re:Writing Speed-of-Light Flash Attention for 5090 in CUDA C++ by neilmovva on Saturday August 23, 2025 @02:52PM (#44998158)

Not really:

5090: 210 TF / $2k == 105 TF/$k

B200: 2250 TF / $40k == 56 TF/$k

Getting only 2x the FLOPs per dollar probably isn't worth the hassle of having to rack 10x as many GPUs, while having no NVLink.

[ Parent | on HN ]

Re:Writing Speed-of-Light Flash Attention for 5090 in CUDA C++ by lossolo on Saturday August 23, 2025 @04:15PM (#44998778)

One of the reasons they removed NVLink from consumer cards (they supported it before). There’s also an issue with power consumption (1xB200 vs 10x5090)

[ Parent | on HN ]

Re:Writing Speed-of-Light Flash Attention for 5090 in CUDA C++ by steinvakt2 on Saturday August 23, 2025 @08:04PM (#45000113)

Sure, but when spending 20x more, getting almost twice the compute per buck seems expected

[ Parent | on HN ]