News for hackers. Stuff that matters.
Re:Flavors of SIMD by dragontamer on Wednesday February 20, 2019 @11:43AM (#19209325)
> As soon as you have lots of reductions, etc. where threads will wait a CPU will also be a much better choice.

Two things:

1. Reductions have been implemented efficiently on GPUs for a while now. See this implementation of Parallel Prefix Scan https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch39....

That right there is your standard "reduce" algorithm on a GPU. Its work-efficient and utilization of the GPU remains very high.

2. GPUs have a ridiculously efficient "barrier" instruction. In fact, GPU "Barriers" are implemented at the assembly level on GPUs. Thread synchronization is actually very, very, very cheap on GPUs. In fact, if your workgroup matches the wavefront / warp of the underlying GPU, then a barrier is simply a NOP: as cheap as it gets.

The main issue with GPUs is that they are a bit mysterious right now. Not a lot of people program them. But the more I work with them, the more I realize how flexible and fast their synchronization tools are.

There are no comments on this comment yet.