The conceptually independent threads are executed on 32 wide SIMD cores. "All the cores" within a warp/wavefront must execute the same instruction each cycle.

GPU threads got even more independent in nvidia'a volta architecture - search Volta ITS for details.