News for hackers. Stuff that matters.
Re:Show HN: Flash Attention in ~100 lines of CUDA by winwang on Saturday March 16, 2024 @05:56PM (#39729842)
Here's an article on syncing in CUDA via cooperative groups: https://developer.nvidia.com/blog/cooperative-groups/

There's also explicit warp synchronization, i.e. __syncwarp(). More on warp primitives here: https://developer.nvidia.com/blog/using-cuda-warp-level-prim...

There are no comments on this comment yet.