They're treated as independent compute units and the host program load-balances compute work between them. You can even do P2P memory copy or access between your CUDA devices without going through the CPU. See https://www.nvidia.com/docs/IO/116711/sc11-multi-gpu.pdf for a high-level overview.