It's true that the programming model allows that, but underneath all threads within the warp will execute the same instructions. However if there is a branch, some threads can be predicated so their instructions have no effect. This is called warp divergence and can become a performance issue. If possible branch only on threadidx using multiples of the warp size. There's a cool slide deck on implementing a parallel sum algorithm that explains this really well.

Do you have any idea what the source of the slide deck could have been? It sounds very interesting and I'd love to see it.

Sure! I meant this one: https://developer.download.nvidia.com/assets/cuda/files/redu...

Yes but can you do that from Julia without writing CUDA/OpenCL code?

I'm happy that Julia supports GPU programming for simple code, but I don't see how you can run algorithms with inter-thread communication.

That's how you do it:

https://github.com/JuliaGPU/GPUArrays.jl/blob/master/src/lin...

Or directly with cuda shfl intrinsics: https://github.com/JuliaGPU/CUDAnative.jl/blob/b249dfd145501...

Oh cool! Thanks!

Yes you can.